Decoding device and decoding method, and coding device and coding method

ABSTRACT

There is provided a decoding device including circuitry configured to receive coded data and conversion information, the coded data pertaining to an image having luminance in a first dynamic range and the conversion information pertaining to a conversion of dynamic range of the luminance of the image from the first dynamic range into a second dynamic range; and decode the received coded data so as to generate the image, wherein the conversion uses a knee function.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.14/491,539 (filed on Sep. 19, 2014), which claims priority JapanesePatent Application nos. 2013-215060 (filed on Oct. 15, 2013),2013-272945 (filed on Dec. 27, 2013), and 2014-042174 (filed on Mar. 4,2014), the entire contents of each of which are incorporated herein byreference.

TECHNICAL FIELD

The present disclosure relates to a decoding device and a decodingmethod, and a coding device and a coding method, and particularly to adecoding device and a decoding method, and a coding device and a codingmethod capable of converting a decoded image into a desired image with adifferent dynamic range.

BACKGROUND ART

In recent years, apparatuses which conform to a method such as MovingPicture Experts Group (MPEG) have been widely spread for both ofinformation delivery in broadcasting stations or the like andinformation reception in ordinary homes. MPEG compresses the imageinformation through orthogonal transform such as discrete cosinetransform and motion compensation by using redundancy unique to theimage information.

Particularly, an MPEG2 (ISO/IEC 13818-2) method is defined as a generaluse image coding method, and is currently widely used in extensiveapplications for professional use and consumer use as a standardcovering both an interlaced scanning image and a progressive scanningimage, and a standard resolution image and a high definition image. Bythe use of the MPEG2 method, it is possible to realize a highcompression ratio and good image quality, for example, by assigning abit rate of 4 Mbps to 8 Mbps to an interlaced scanning image of astandard resolution having 720×480 pixels and assigning a bit rate of 18Mbps to 22 Mbps to an interlaced scanning image of a high resolutionhaving 1920×1088 pixels.

MPEG2 has mainly targeted high image quality coding suitable forbroadcasting, but has not handled a coding method at a bit rate lowerthan that in MPEG1, that is, at a higher compression ratio. With thewide use of portable terminals, the desire for such a coding method hasbeen considered to increase, and thus an MPEG4 coding method has beenstandardized so as to correspond thereto. In relation to an image codingmethod of MPEG4, a standard thereof was approved as an internationalstandard entitled ISO/IEC 14496-2 in December 1998.

In addition, in recent years, standardization of a standard called H.26L(ITU-T Q6/16 VCEG) has progressed for the original purpose of imagecoding for video conference use. H.26L uses a larger calculation amountdue to coding and decoding than the coding method of the related artsuch as MPEG2 or MPEG4, but is known for realizing higher codingefficiency.

Further, as part of activities of MPEG4, Joint Model ofEnhanced-Compression Video Coding is currently being standardized inorder to realize higher coding efficiency by also incorporatingfunctions which are not supported by H.26L, on the basis of H.26L. Asfor the standardization schedule thereof, the coding method has becomean international standard under the name of H.26L and MPEG-4 Part 10((Advanced Video Coding (AVC)) in March 2003.

In addition, as an extension of the AVC method, Fidelity Range Extension(FRExt) which includes coding tools for use in business such as RGB orYUS422 and YUV444 and also includes 8×8 DCT or quantization matrixdefined in MPEG2 was standardized in February 2005. This realizes acoding method in which even film noise included in a movie can befavorably expressed by using the AVC method, and thus leads to use forvarious applications such as a Blu-Ray (registered trademark) disc (BD).

However, recently, there have been increasing demands for highercompression ratio coding, such as a demand for compression of an imagewith about 4000×2000 pixels which is four times the size of ahigh-vision image or a demand for delivery of a high-vision image inlimited transmission capacity circumstances such as the Internet. Forthis reason, study of improvement of coding efficiency is beingcurrently performed in Video Coding Expert Group (VCEG) affiliated tothe above ITU-T.

In addition, currently, for the purpose of improvement in higher codingefficiency than that of AVC, standardization of a coding method calledHigh Efficiency Video Coding (HEVC) is in progress by JointCollaboration Team-Video Coding (JCTVC) which is a joint standardizationorganization of ITU-T and ISO/IEC. NPL 1 has been currently published asa draft in August 2013.

Meanwhile, recently, with the progress of techniques, a high dynamicrange (HDR) display with the maximum luminance of 500 nit or 1000 nithas been started to be sold on the market.

In a case where a standard dynamic range (SDR) display and an HDRdisplay are mixed, it is necessary to encode each of an SDR image and anHDR image in the AVC method or the HEVC method, and thus a data amountincreases. Therefore, a method is considered in which one of the SDRimage and the HDR image is coded, and then a dynamic range is convertedafter decoding is performed as necessary, thereby generating the other.

CITATION LIST Non Patent Literature

[NPL 1]

-   Benjamin Bross, Gary J. Sullivan, Ye-Kui Wang, “Editors' proposed    corrections to HEVC version 1”, JCTVC-M0432_v3, 2013. 4.18-4.26

SUMMARY Technical Problem

However, conversion into an image which is intended by a producer is notconsidered when conversion of a dynamic range is converted.

It is desirable to convert a decoded image into a desired image with adifferent dynamic range.

Solution to Problem

According to an embodiment of the present disclosure, there is provideda decoding device including: circuitry configured to receive coded dataand conversion information, the coded data pertaining to an image havingluminance in a first dynamic range and the conversion informationpertaining to a conversion of dynamic range of the luminance of theimage from the first dynamic range into a second dynamic range; anddecode the received coded data so as to generate the image, wherein theconversion uses a knee function.

A decoding method of causing a decoding device to perform: receivingcoded data and conversion information, the coded data pertaining to animage having luminance in a first dynamic range and the conversioninformation pertaining to a conversion of dynamic range of the luminanceof the image from the first dynamic range into a second dynamic range;and decoding the received coded data so as to generate the image,wherein the conversion uses a knee function.

A coding device including: circuitry configured to set conversioninformation pertaining to a conversion of dynamic range of a luminanceof an image from a first dynamic range into a second dynamic range; andcode the image having luminance in the first dynamic range so as togenerate coded data, wherein the conversion uses a knee function.

A non-transitory computer-readable medium having stored thereon codeddata and conversion information, the coded data pertaining to an imagehaving luminance in a first dynamic range and the conversion informationpertaining to a conversion of dynamic range of the luminance of theimage from the first dynamic range into a second dynamic range, whereina decoding device decodes coded data, generates the image based on thedecoded data, and converts the dynamic range based on the conversioninformation including a knee point.

According to an embodiment of the present disclosure, there is provideda decoding device including an extraction unit that extracts coded dataand conversion information from a coded stream including the coded dataof a first image which is an image having luminance in a first dynamicrange and the conversion information regarding conversion of a dynamicrange of the luminance of the image from the first dynamic range into asecond dynamic range; and a decoding unit that decodes the coded dataextracted by the extraction unit so as to generate the first image.

A decoding method according to an embodiment of the present disclosurecorresponds to the decoding device according to the embodiment of thepresent disclosure.

According to an embodiment of the present disclosure, coded data andconversion information are extracted from a coded stream including thecoded data of a first image which is an image having luminance in afirst dynamic range and the conversion information which is informationregarding conversion of a dynamic range of the luminance of the imagefrom the first dynamic range into a second dynamic range, and theextracted coded data is decoded so that the first image is generated.

According to another embodiment of the present disclosure, there isprovided a coding device including a setting unit that sets conversioninformation which is information regarding conversion of a dynamic rangeof luminance of an image from a first dynamic range into a seconddynamic range; a coding unit that codes a first image which is the imagehaving luminance in the first dynamic range so as to generate codeddata; and a transmission unit that transmits a coded stream includingthe conversion information set by the setting unit and the coded data ofthe first image generated by the coding unit.

A coding method of another embodiment of the present disclosurecorresponds to the coding device according to another embodiment of thepresent disclosure.

According to an embodiment of the present disclosure, conversioninformation is set which is information regarding conversion of adynamic range of luminance of an image from a first dynamic range into asecond dynamic range, a first image which is the image having luminancein the first dynamic range is coded so that coded data is generated, anda coded stream including the conversion information and the coded dataof the first image is transmitted.

In addition, the decoding device and the coding device according to theembodiments may be implemented by executing a program in a computer.

Further, the program executed in the computer in order to implement thedecoding device and the coding device according to an embodiment may beprovided by transmitting the program via a transmission medium or byrecording the program on a recording medium.

The decoding device and the coding device according to embodiments maybe standalone devices, and may be an internal block forming a singleapparatus.

Advantageous Effects of Invention

According to an embodiment of the present disclosure, it is possible todecode coded data of an image. In addition, according to the embodimentof the present disclosure, it is possible to convert a decoded imageinto a desired image with a different dynamic range.

According to another embodiment of the present disclosure, it ispossible to code an image. In addition, according to another embodimentof the present disclosure, it is possible to code an image so that adecoded image can be converted into a desired image with a differentdynamic range during decoding.

In addition, the effects described here are not necessarily limited, andthere may be any one of effects described in the present disclosure.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram illustrating an SDR image.

FIG. 2 is a diagram illustrating an HDR image.

FIG. 3 is a diagram illustrating an overview of coding in an embodimentof the present disclosure.

FIG. 4 is a diagram illustrating an overview of decoding in anembodiment of the present disclosure.

FIG. 5 is a diagram illustrating knee decompression.

FIG. 6 is a block diagram illustrating a configuration example of anembodiment of a coding device to which the present disclosure isapplied.

FIG. 7 is a diagram illustrating an example of syntax ofknee_function_info SEI.

FIG. 8 is a diagram illustrating each piece of information set in theknee_function_info SEI of FIG. 7.

FIG. 9 is a diagram illustrating an example of conversion informationset in the knee_function_info SEI.

FIG. 10 is a diagram illustrating an example of conversion informationset in the knee_function_info SEI.

FIG. 11 is a flowchart illustrating a stream generation processperformed by the coding device.

FIG. 12 is a block diagram illustrating a configuration example of anembodiment of a decoding device to which the present disclosure isapplied.

FIG. 13 is a flowchart illustrating an image generation processperformed by the decoding device of FIG. 12.

FIG. 14 is a diagram illustrating another example of syntax of theknee_function_info SEI.

FIG. 15 is a diagram illustrating each piece of information set in theknee_function_info SEI of FIG. 14.

FIG. 16 is a block diagram illustrating a configuration example of afirst embodiment of a coding device to which an embodiment of thepresent disclosure is applied.

FIG. 17 is a diagram illustrating a first example of syntax ofknee_function_info SEI set by a setting unit of FIG. 16.

FIG. 18 is a diagram illustrating each piece of information set in theknee_function_info SEI of FIG. 17.

FIG. 19 is a diagram illustrating an example of DR conversioninformation of FIG. 17.

FIG. 20 is a diagram illustrating an example of DR conversioninformation of FIG. 17.

FIG. 21 is a flowchart illustrating a stream generation processperformed by the coding device of FIG. 16.

FIG. 22 is a block diagram illustrating a configuration example of asecond embodiment of a decoding device to which the present disclosureis applied.

FIG. 23 is a flowchart illustrating an image generation processperformed by the decoding device of FIG. 22.

FIG. 24 is a diagram illustrating another example of DR conversioninformation of FIG. 17.

FIG. 25 is a diagram illustrating still another example of DR conversioninformation of FIG. 17.

FIG. 26 is a diagram illustrating an example of syntax oftone_mapping_info_SEI including the DR conversion information of FIG.17.

FIG. 27 is a diagram illustrating another example of syntax oftone_mapping_info_SEI including the DR conversion information of FIG.17.

FIG. 28 is a diagram illustrating a second example of syntax ofknee_function_info SEI set by the setting unit of FIG. 16.

FIG. 29 is a diagram illustrating each piece of information set in theknee_function_info SEI of FIG. 28.

FIG. 30 is a diagram illustrating an example of DR conversioninformation of FIG. 28.

FIG. 31 is a diagram illustrating an example of DR conversioninformation of FIG. 28.

FIG. 32 is a diagram illustrating an example of syntax oftone_mapping_info_SEI including the DR conversion information of FIG.28.

FIG. 33 is a diagram illustrating each piece of information set in theknee_function_info SEI of FIG. 28 in a case where the number of kneepoints is restricted.

FIG. 34 is a diagram illustrating an example of the knee_function_infoSEI of FIG. 28 in a case where the number of knee points is restricted.

FIG. 35 is a diagram illustrating an example of thetone_mapping_info_SEI of FIG. 32 in a case where the number of kneepoints is restricted.

FIG. 36 is a diagram illustrating a third example of syntax ofknee_function_info SEI set by the setting unit of FIG. 16.

FIG. 37 is a diagram illustrating each piece of information set in theknee_function_info SEI of FIG. 36.

FIG. 38 is a diagram illustrating an example of DR conversioninformation of FIG. 36.

FIG. 39 is a diagram illustrating an example of syntax oftone_mapping_info_SEI including the DR conversion information of FIG.36.

FIG. 40 is a diagram illustrating a fourth example of syntax ofknee_function_info SEI set by the setting unit of FIG. 16.

FIG. 41 is a diagram illustrating each piece of information set in theknee_function_info SEI of FIG. 40.

FIG. 42 is a diagram illustrating an example of DR conversioninformation of FIG. 40.

FIG. 43 is a diagram illustrating an example of DR conversioninformation of FIG. 40.

FIG. 44 is a diagram illustrating an operation of the decoding device ina case where the knee_function_info SEI of FIG. 40 is set in aplurality.

FIG. 45 is a diagram illustrating an example of syntax oftone_mapping_info_SEI including the DR conversion information of FIG.40.

FIG. 46 is a diagram illustrating a box of MP4 in which DR conversioninformation is disposed.

FIG. 47 is a diagram illustrating an example of syntax of ToneMapInfo.

FIG. 48 is a diagram illustrating that semantics in a firstconfiguration of a third embodiment of a coding device to which thepresent disclosure is applied is different from that in the secondembodiment.

FIG. 49 is a block diagram illustrating a first configuration example ofan embodiment of a decoding system to which the present disclosure isapplied.

FIG. 50A is a diagram illustrating an example of a knee point and afunction of knee conversion defined by knee_function_info SEI which isreceived by the decoding system of FIG. 49.

FIG. 50B is a diagram illustrating an example of a knee point and afunction of knee conversion defined by knee_function_info SEI which isreceived by the decoding system of FIG. 49.

FIG. 51 is a diagram illustrating an example of an approximate functionof the knee conversion of FIG. 50.

FIG. 52 is a diagram illustrating an example of an approximate functionof the knee conversion of FIG. 50.

FIG. 53 is a flowchart illustrating a decoding process performed by thedecoding device of FIG. 49.

FIG. 54 is a flowchart illustrating a display process performed by adisplay device of FIG. 49.

FIG. 55 is a diagram illustrating an example of syntax ofknee_function_info SEI in a second configuration of the third embodimentof the coding device to which the present disclosure is applied.

FIG. 56 is a diagram illustrating a difference in semantics of FIG. 55from the second embodiment.

FIG. 57A is a diagram illustrating an example of a knee point and afunction of knee conversion defined by knee_function_info SEI of FIG.55.

FIG. 57B is a diagram illustrating an example of a knee point and afunction of knee conversion defined by knee_function_info SEI of FIG.55.

FIG. 58 is a diagram illustrating an example of an approximate functionof the knee conversion of FIG. 57.

FIG. 59 is a diagram illustrating an example of an approximate functionof the knee conversion of FIG. 57.

FIG. 60 is a diagram illustrating an example of syntax ofapproximate_knee_function_info SEI.

FIG. 61 is a diagram illustrating a relationship between an inputelectrical signal and display luminance of a CRT.

FIG. 62 is a diagram illustrating an electrical signal which isproportional to luminance.

FIG. 63 is a diagram illustrating a relationship between an inputelectrical signal and display luminance.

FIG. 64 is a diagram illustrating a function with a characteristicreverse to a function of FIG. 61.

FIG. 65 is a diagram illustrating an example of a flow of a processuntil an image is displayed from capturing of the image.

FIG. 66 is a diagram illustrating OETF for use in an SDR image.

FIG. 67 is a diagram illustrating OETF for use in an HDR image.

FIG. 68 is a diagram illustrating an overview of a photoelectricconversion process in a fourth embodiment.

FIG. 69 is a diagram illustrating an overview of an electro-opticalconversion in the fourth embodiment.

FIG. 70 is a block diagram illustrating a configuration example of thefourth embodiment of a coding device to which the present disclosure isapplied.

FIG. 71 is a flowchart illustrating a stream generation processperformed by the coding device of FIG. 70.

FIG. 72 is a block diagram illustrating a configuration example of thefourth embodiment of a decoding device to which the present disclosureis applied.

FIG. 73 is a flowchart illustrating an image generation processperformed by the decoding device of FIG. 72.

FIG. 74 is a block diagram illustrating a hardware configuration exampleof a computer.

FIG. 75 is a diagram illustrating an example of a multi-viewpoint imagecoding method.

FIG. 76 is a diagram illustrating a configuration example of amulti-view image coding device to which the present disclosure isapplied.

FIG. 77 is a diagram illustrating a configuration example of amulti-view image decoding device to which the present disclosure isapplied.

FIG. 78 is a diagram illustrating an example of a layer image codingmethod.

FIG. 79 is a diagram illustrating an example of spatially scalablecoding.

FIG. 80 is a diagram illustrating an example of a temporally scalablecoding.

FIG. 81 is a diagram illustrating an example of scalable coding of anS/N ratio.

FIG. 82 is a diagram illustrating a configuration example of a layerimage coding device to which the present disclosure is applied.

FIG. 83 is a diagram illustrating a configuration example of a layerimage decoding device to which the present disclosure is applied.

FIG. 84 is a block diagram illustrating a schematic configurationexample of a television apparatus to which the present disclosure isapplied.

FIG. 85 is a block diagram illustrating a schematic configurationexample of a mobile phone to which the present disclosure is applied.

FIG. 86 is a block diagram illustrating a schematic configurationexample of a recording/reproducing apparatus to which the presentdisclosure is applied.

FIG. 87 is a block diagram illustrating a schematic configurationexample of an imaging apparatus to which the present disclosure isapplied.

FIG. 88 is a block diagram illustrating an example of using scalablecoding.

FIG. 89 is a block diagram illustrating another example of usingscalable coding.

FIG. 90 is a block diagram illustrating still another example of usingscalable coding.

FIG. 91 is a diagram illustrating a schematic configuration example of avideo set to which the present disclosure is applied.

FIG. 92 is a diagram illustrating a schematic configuration example of avideo processor to which the present disclosure is applied.

FIG. 93 is a diagram illustrating another schematic configurationexample of a video processor to which the present disclosure is applied.

DESCRIPTION OF EMBODIMENTS

[Base of Present Disclosure]

[Description of SDR Image]

FIG. 1 is a diagram illustrating an SDR image.

As illustrated in FIG. 1, an SDR image is, for example, an image whoseimage quality is adjusted so as to correspond to a display device withthe maximum luminance 100 nit (candela per square meter). Since themaximum luminance in the natural system reaches 20000 nit or more insome cases, in the SDR image, a brightness dynamic range is greatlycompressed.

[Description of HDR Image]

FIG. 2 is a diagram illustrating an HDR image.

As illustrated in FIG. 2, an HDR image is an image in which a dynamicrange of luminance is greater than 0 to 100%. In the presentspecification, unless otherwise described, a dynamic range of aluminance of the HDR image is 0 to 400%. For example, as illustrated inFIG. 2, in a case where an HDR image in which a dynamic range ofluminance is 0 to 800% (800 nit) is coded, and is recorded on a Blu-ray(registered trademark) disc (BD) or the like, attribute informationindicating the luminance is also recorded along with the HDR image. Inaddition, the attribute information is input to a display device alongwith a decoded HDR image, and the HDR image is displayed as an image inwhich a dynamic range of luminance is 0 to 800%.

Further, in a case where the maximum luminance of the display device is1000 nit, for example, luminance of an HDR image is scaled to 1000 nitand is displayed. Even in a case where the scaling is performed in thisway, an HDR image has a dynamic range of luminance of 0 to 800%, andthus image quality deterioration thereof due to the scaling is smallerthan that of an SDR image.

[First Embodiment]

(Overview of Coding in First Embodiment)

FIG. 3 is a diagram illustrating an overview of coding in a firstembodiment of a coding device to which the present disclosure isapplied.

In FIG. 3, the transverse axis expresses a luminance value (input codevalue), and the longitudinal axis expresses luminance (output videolevel). In addition, the luminance value of the transverse axis of FIG.3 is a value obtained by setting the number of bits of the luminancevalue to 10 bits and setting white luminance having undergone kneeconversion to 100%, but a luminance value converted into luminance inpractice is a value which is normalized to 0 or more and 1 or less. Thisis also the same for FIG. 5 described later.

As illustrated in FIG. 3, in the first embodiment, 80% to 400% of an HDRimage in which a dynamic range of luminance is 0 to 400% isknee-compressed to 80% to 100%, so that an SDR image in which a dynamicrange of luminance is 0 to 100% is generated and is then coded.

[Overview of Decoding in First Embodiment]

FIG. 4 is a diagram illustrating an overview of decoding in the firstembodiment of a decoding device to which the present disclosure isapplied.

As illustrated in FIG. 4, in the first embodiment, coded data of the SDRimage in which a dynamic range of luminance is 0 to 100%, generated asdescribed in FIG. 3, is decoded. In a case where a display unit is anSDR display, the SDR image which is obtained as a result of the decodingis input to and is displayed on the display unit without change. On theother hand, in a case where the display unit is an HDR display, the SDRimage obtained as a result of the decoding is scaled to an HDR image,and is input to and is displayed on the display unit.

Specifically, as illustrated in FIG. 5, 80% to 1.00% of the SDR image inwhich a dynamic range of luminance is 0 to 100% is knee-decompressed to80% to 400%, and thus an HDR image in which a dynamic range of luminanceis 0 to 400% is generated. In addition, the generated HDR image isdisplayed.

In addition, at this time, in order to generate a desired HDR image,information regarding conversion from an SDR image into the desired HDRimage, such as a range (80% to 100% in the example of FIG. 5) ofluminance of an SDR image which is knee-decompressed, and a range (80%to 400% in the example of FIG. 5) of luminance of an HDR imagecorresponding to the range, is necessary. Therefore, in the firstembodiment, conversion information regarding conversion from an SDRimage into an HDR image is transmitted from the coding device to thedecoding device, and thus a desired HDR image can be generated from adecoded SDR image in the decoding device.

(Configuration Example of First Embodiment of Coding Device)

FIG. 6 is a block diagram illustrating a configuration example of thefirst embodiment of a coding device to which the present disclosure isapplied.

A coding device 10 of FIG. 6 includes a setting unit 11, a coding unit12, a transmission unit 13, and a conversion unit 14, and codes an SDRimage which is converted from an HDR image in a method conforming to theHEVC method.

Specifically, the setting unit 11 of the coding device 10 sets asequence parameter set (SPS), a picture parameter set (PPS), VUI, andthe like. In addition, the setting unit 11 sets knee_function_infoSupplemental Enhancement Information (SEI) including conversioninformation in response to a command from a user (producer). The settingunit 11 supplies the parameter sets including the set SPS, PPS, VUI,knee_function_info SEI, and the like to the coding unit 12.

The coding unit 12 codes the SDR image supplied from the conversion unit14 in the HEVC method. The coding unit 12 generates a coded stream fromcoded data which is obtained as a result of the coding and the parametersets which are supplied from the setting unit 11, and transmits thegenerated coded stream to the transmission unit 13.

The transmission unit 13 transmits the coded stream supplied from thecoding unit 12, to a decoding device described later. In addition, thetransmission unit 13 may transmit the coded stream to a recording devicewhich records the coded stream on a recording medium such as a BD. Inthis case, the coded stream is transmitted to the decoding device viathe recording medium.

The conversion unit 14 converts an HDR image input from an externaldevice into an SDR image through knee compression, and supplies the SDRimage to the coding unit 12.

(Example of Syntax of Knee_Function_Info SEI)

FIG. 7 is a diagram illustrating an example of syntax ofknee_function_info SEI, and FIG. 8 is a diagram illustrating each pieceof information set in the knee_function_info SEI of FIG. 7.

As illustrated in FIG. 7, input knee position information(knee_point_of_input), output knee position information(knee_point_of_input), output luminance range information(output_white_level_range), output luminance information(output_white_level_range_luminace), and the like are set in theknee_function_info SEI as conversion information.

The input knee position information is information indicating theminimum value (knee point) of luminance which is knee decompression ofan SDR image which is an unconverted image. The input knee positioninformation is a permillage of a knee point when the maximum value ofluminance of an SDR image is set to 1000 permil.

The output knee position information is information indicating luminanceof an HDR image which is a converted image, corresponding to the minimumvalue (knee point) of luminance which is a knee decompression target ofan SDR image which is an unconverted image. The output knee positioninformation is a permillage of luminance corresponding to a knee pointwhen the maximum value of luminance of an HDR image is set to 1000permil.

The output luminance range information is information indicating whiteluminance of an HDR image which is a converted image. In addition, theoutput luminance information is information indicating brightness(luminance) of the display unit, corresponding to white of the HDR imagewhich is a converted image.

(Example of Conversion Information)

FIGS. 9 and 10 are diagrams illustrating examples of conversioninformation set in the knee_function_info SEI.

In the example of FIG. 9, the user sets an HDR image which is obtainedas a result of knee-decompressing 80% to 100% of luminance of an SDRimage to 80% to 400% is used as a desired HDR image. In this case, inthe knee_function_info SEI, 800 as the input knee position information(knee_point_of_input), and 200 is set as the output knee positioninformation (knee_point_of_output).

Therefore, a decoding device described later can knee-decompress 80% to100% of luminance of an SDR image which is obtained as a result ofdecoding to 80% to 400% on the basis of the input knee positioninformation and the output knee position information. As a result, thedecoding device can convert the SDR image obtained as a result of thedecoding into a desired HDR image.

In addition, in the example of FIG. 9, the output luminance rangeinformation (output_white_level_range) is 400, and the output luminanceinformation (output_white_level_range_luminace) is 800 (candela persquare meter).

In the example of FIG. 10, the user sets an HDR image which is obtainedas a result of knee-decompressing 80% to 100% of luminance of an SDRimage to 100% to 400% as a desired HDR image. In this case, in theknee_function_info SEI, 800 is set as the input knee positioninformation (knee_point_of_input), and 250 is set as the output kneeposition information (knee_point_of_output).

Therefore, the decoding device described later can knee-decompress 80%to 100% of luminance of an SDR image which is obtained as a result ofdecoding to 100% to 400% on the basis of the input knee positioninformation and the output knee position information. As a result, thedecoding device can convert the SDR image obtained as a result of thedecoding into a desired HDR image.

In addition, in the example of FIG. 10, the output luminance rangeinformation (output_white_level_range) is 400, and the output luminanceinformation (output_white_level_range_luminace) is 800 (candela persquare meter).

<Description of Process in Coding Device>

FIG. 11 is a flowchart illustrating a stream generation processperformed by the coding device 10.

In step S10 of FIG. 11, the conversion unit 14 of the coding device 10converts an HDR image which is input from an external device, into anSDR image which is then supplied to the coding unit 12.

In step S11, the setting unit 11 sets an SPS. In step S12, the settingunit 11 sets VUI. In step S13, the setting unit 11 sets a PPS.

In step S14, the setting unit 11 sets knee_function_info SEI in responseto an instruction or the like from a user. The setting unit 11 suppliesthe parameter sets including the set SPS, PPS, VUI, knee_function_infoSEI, and the like to the coding unit 12.

In step S15, the coding unit 12 codes the SDR image supplied from theconversion unit 14 in the HEVC method. In step S16, the coding unit 12generates a coded stream from coded data which is obtained as a resultof the coding and the parameter sets which are supplied from the settingunit 11, and transmits the generated coded stream to the transmissionunit 13.

In step S17, the transmission unit 13 transmits the coded streamsupplied from the coding unit 12, to the decoding device describedlater, and then finishes the process.

As mentioned above, the coding device 10 sets and transmitsknee_function_info SEI including conversion information, and thus thedecoding device described later can convert an SDR image obtained as aresult of decoding into a desired HDR image on the basis of theconversion information. Therefore, it can be said that the coding device10 can code an SDR image so that a decoded SDR image can be convertedinto a desired HDR image during decoding.

In addition, since the conversion information is set, the coding device10 can generate a coded stream of an image corresponding to an HDRdisplay and an SDR display only by coding an SDR image. Therefore, it ispossible to further reduce a data amount of a coded stream than in acase of coding both an HDR image and an SDR image.

(Configuration Example of First Embodiment of Decoding Device)

FIG. 12 is a block diagram illustrating a configuration example of anembodiment of a decoding device which decodes a coded stream transmittedfrom the coding device 10 of FIG. 6 and to which the present disclosureis applied.

A decoding device 50 of FIG. 12 includes a reception unit 51, anextraction unit 52, a decoding unit 53, a conversion unit 54, a displaycontrol unit 55, and a display unit 56.

The reception unit 51 of the decoding device 50 receives the codedstream transmitted from the coding device 10 of FIG. 6, and supplies thecoded stream to the extraction unit 52.

The extraction unit 52 extracts the parameter sets and the coded data ofthe SDR image from the coded stream which is supplied from the receptionunit 51. The extraction unit 52 supplies the parameter sets and thecoded data to the decoding unit 53. In addition, the extraction unit 52supplies the knee_function_info SEI among the parameter sets, to theconversion unit 54.

The decoding unit 53 decodes the coded data of the SDR image suppliedfrom the extraction unit 52 in the HEVC method. At this time, thedecoding unit 53 also refers to the parameter sets supplied from theextraction unit 52 as necessary. The decoding unit 53 supplies the SDRimage which is obtained as a result of decoding to the conversion unit54.

The conversion unit 54 converts the SDR image supplied from the decodingunit 53 into an HDR image through knee decompression on the basis of theconversion information included in the knee_function_info SEI suppliedfrom the extraction unit 52, and supplies the HDR image to the displaycontrol unit 55.

The display control unit 55 displays the HDR image supplied from theconversion unit 54 on the display unit 56. The display unit 56 is an HDRdisplay.

<Description of Process in Decoding Device>

FIG. 13 is a flowchart illustrating an image generation processperformed by the decoding device 50 of FIG. 12.

In step S51 of FIG. 13, the reception unit 51 of the decoding device 50receives the coded stream transmitted from the coding device 10 of FIG.6, and supplies the coded stream to the extraction unit 52.

In step S52, the extraction unit 52 extracts the parameter sets and thecoded data of the SDR image from the coded stream which is supplied fromthe reception unit 51. The extraction unit 52 supplies the parametersets and the coded data of the SDR image to the decoding unit 53. Inaddition, the extraction unit 52 supplies the knee_function_info SEIamong the parameter sets, to the conversion unit 54.

In step S53, the decoding unit 53 decodes the coded data of the SDRimage supplied from the extraction unit 52 in the HEVC method. At thistime, the decoding unit 53 also refers to the parameter sets suppliedfrom the extraction unit 52 as necessary. The decoding unit 53 suppliesthe SDR image which is obtained as a result of decoding to theconversion unit 54.

In step S54, the conversion unit 54 acquires the conversion informationfrom the knee_function_info SEI which is supplied from the extractionunit 52.

In step S55, the conversion unit 54 converts the SDR image supplied fromthe decoding unit 53 into an HDR image on the basis of the conversioninformation, and supplies the HDR image to the display control unit 55.

In step S56, the display control unit 55 displays the HDR image suppliedfrom the conversion unit 54 on the display unit 56, and finishes theprocess.

As mentioned above, the decoding device 50 converts the SDR imageobtained as a result of decoding into the HDR image on the basis of theconversion information, and thus can convert the SDR image obtained as aresult of decoding into a desired HDR image.

(Another Example of Syntax of Knee_Function_Info SEI)

FIG. 14 is a diagram illustrating another example of syntax ofknee_function_info SEI, and FIG. 15 is a diagram illustrating each pieceof information set in the knee_function_info SEI of FIG. 14.

The knee_function_info SEI of FIG. 14 is the same as theknee_function_info SEI of FIG. 7 except that luminance range information(white_level_range) and luminance information (white_levelrange_luminance) are set instead of the output luminance rangeinformation (output_white_level_range) and the output luminanceinformation (output_white_level_range_luminance).

The luminance range information is output luminance range informationwhen input knee position information (knee_point_of_input) is equal toor more than output knee position information (knee_point_of output),that is, when knee decompression is performed on a decoding side in thesame manner as in the first embodiment.

On the other hand, when the input knee position information is less thanthe output knee position information, that is, when knee compression isperformed on the decoding side, the luminance range information isinformation indicating white luminance of an unconverted image (forexample, an HDR image).

Similarly, the luminance information (white_level_range_luminance) isoutput luminance information when input knee position information isequal to or more than output knee position information in the samemanner as in the first embodiment, and is information indicating whiteluminance (value) of an unconverted image (for example, an HDR image)when the input knee position information is less than the output kneeposition information.

In addition, in the first embodiment, only an SDR image is coded in thecoding device 10, but only an HDR image converted from the SDR image maybe coded. In this case, information regarding conversion from the SDRimage into the HDR image is set in SEI and is transmitted to thedecoding device 50. Specifically, the knee_function_info SEI illustratedin FIG. 7 or FIG. 15 in which an unconverted image is set as an HDRimage, and a converted image is set as an SDR image is transmitted tothe decoding device 50. In addition, the decoding device 50 converts anHDR image into an original SDR image with high accuracy on the basis ofthe knee_function_info SEI.

In addition, in the first embodiment, the display unit 56 is an HDRdisplay, but the display unit 56 may be an SDR display. In this case,the conversion unit 54 supplies an SDR image to the display control unit55 without conversion into an HDR image. Accordingly, the SDR image isdisplayed on the display unit 56.

In addition, a desired image may be an HDR image which is input to thecoding device 10.

In addition, in the first embodiment, the coding device 10 converts anHDR image which is input from an external device into an SDR image whichis then coded, but may code an SDR image which is input from theexternal device without conversion.

[Second Embodiment]

(Configuration Example of Second Embodiment of Coding Device)

FIG. 16 is a block diagram illustrating a configuration example of asecond embodiment of a coding device to which the present disclosure isapplied.

Among constituent elements illustrated in FIG. 16, the same constituentelements as the constituent elements of FIG. 6 are given the samereference numeral. Repeated description will be omitted as appropriate.

A configuration of a coding device 70 of FIG. 16 is different from theconfiguration of the coding device 10 of FIG. 6 in that a setting unit71, a coding unit 72, and a conversion unit 73 are provided instead ofthe setting unit 11, the coding unit. 12, and the conversion unit 14.The coding device 70 codes an HDR image which is input from an externaldevice, or codes an SDR image which is converted from an HDR image, in amethod conforming to the HEVC method.

Specifically, the setting unit 71 of the coding device 70 sets, an SPS,a PPS, VUI, and the like. In addition, the setting unit 71 sets SEI suchas knee_function_info SEI including DR conversion information inresponse to a command from a user (producer). The DR conversioninformation is information regarding conversion from a dynamic range ofluminance of an image which is a coding target into a different dynamicrange. The setting unit 71 supplies the parameter sets including the setSPS, PPS, VUI, knee_function_info SEI, and the like to the coding unit72.

The coding unit 72 sets an HDR image or an SDR image supplied from theconversion unit 73 as a coding target image, and codes the coding targetimage in the HEVC method. The coding unit 72 generates a coded streamfrom coded data which is obtained as a result of the coding and theparameter sets which are supplied from the setting unit 71, andtransmits the generated coded stream to the transmission unit 13.

The conversion unit 73 knee-compresses luminance of an HDR image whichis input from an external device so as to generate an SDR image which isthen supplied to the coding unit 72, or supplies an HDR image which isinput from the external device to the coding unit 72 withoutcompression.

(First Example of Syntax of Knee_Function_Info SEI)

FIG. 17 is a diagram illustrating a first example of syntax ofknee_function_info SEI set by the setting unit 71 of FIG. 16, and FIG.18 is a diagram illustrating each piece of information set in theknee_function_info SEI of FIG. 17.

As illustrated in FIG. 17, a knee conversion ID (knee_function_id) and aknee conversion cancel flag (knee_function_cancel_flag) are set in theknee_function_info SEI.

The knee conversion ID is an ID unique to the purpose of knee conversionwhich is knee compression or knee decompression as illustrated in FIG.18. In addition, the knee conversion cancel flag is a flag illustratingwhether or not persistence of previous knee_function_info SEI iscanceled. The knee conversion cancel flag is set to 1 when indicatingthat persistence of previous knee_function_info SEI is canceled, and isset to 0 when the persistence is not canceled.

If the knee conversion cancel flag is 0, as illustrated in FIG. 17, asingle piece of pre-conversion position information (input_knee_point),a single piece of post-conversion position information(output_knee_point), HDR luminance range information (d_range), anddisplay luminance information (d_range_disp_luminance) are set in theknee_function_info SEI as DR conversion information.

The pre-conversion position information is information indicating a kneepoint of a coding target image which is an unconverted image inconversion corresponding to the DR conversion information, and is apermillage of a knee point when the maximum value of luminance of thecoding target image is set to 1000 permil. The knee point is luminance(which is a value obtained by normalizing linear RGB values in the rangeof 0.0 to 1.1) other than 0 which is a start point of a range ofluminance which is knee-converted at the same conversion ratio as thatof a dynamic range of luminance of the coding target image.

The post-conversion position information is information indicating astart point of a range of luminance corresponding to a range ofknee-converted luminance which has a knee point as a start point in animage after being converted (hereinafter, referred to as a convertedimage) in conversion corresponding to the DR conversion information.Specifically, the post-conversion position information is a permillageof luminance of a converted image corresponding to a knee point when themaximum value of luminance of the converted image is set to 1000 permil.

The HDR luminance range information is information indicating apermillage of the maximum value of luminance of an HDR image which is acoding target image or a converted image. In addition, the displayluminance information is information indicating an expected value ofbrightness (luminance) of the display unit corresponding to the maximumvalue of luminance of an HDR image.

(First Example of DR Conversion Information)

FIGS. 19 and 20 are diagrams illustrating examples of the DR conversioninformation set in the knee_function_info SEI of FIG. 17.

In the example of FIG. 19, a coding target image is an SDR image, and auser sets an HDR image which is obtained as a result ofknee-decompressing 80% to 100% of luminance of the SDR image to 80% to400%, as a desired converted image. In this case, in theknee_function_info SEI, 800 is set as the pre-conversion positioninformation (input_knee_point), and 200 is set as the post-conversionposition information (output_knee_point).

In addition, in the example of FIG. 19, the HDR luminance rangeinformation (d_range) is 4000, and the display luminance rangeinformation (d_range_disp_luminance) is 800 (candela per square meter).

As in the case of FIG. 19, in a case where a coding target image is anSDR image, and a converted image is an HDR image, a knee pointinput_knee_point_PER (%) and luminance output_knee_point_PER (%) of aconverted image corresponding to the knee point are defined by thefollowing Equation (1).

$\begin{matrix}{\lbrack {{Math}.\mspace{14mu} 1} \rbrack\mspace{644mu}} & \; \\{{{{input\_ knee}{\_ point}{\_ DR}} = {100 \times \frac{{input\_ knee}{\_ point}}{1000}}}{{{output\_ knee}{\_ point}{\_ DR}} = {\frac{d\_ range}{10} \times \frac{{output\_ knee}{\_ point}}{1000}}}} & (1)\end{matrix}$

Therefore, a decoding device described later recognizes that the kneepoint input_knee_point_PER and the luminance output_knee_point_PER are80% according to Equation (1). In addition, the decoding devicedescribed later recognizes that knee conversion corresponding to the DRconversion information is knee decompression since the pre-conversionposition information is equal to or more than the post-conversionposition information. Further, the decoding device described laterrecognizes that the maximum value of luminance of the converted image is400% from the HDR luminance range information.

As mentioned above, the decoding device described laterknee-decompresses 80% to 100% of luminance of the SDR image which isobtained as a result of decoding to, 80% to 400%. Therefore, thedecoding device can convert the SDR image obtained as a result ofdecoding into a desired HDR image.

In the example of FIG. 20, a coding target image is an HDR image, andthe user sets an SDR image which is obtained as a result ofknee-compressing 80% to 400% of luminance of the HDR image to 80% to100%, as a desired converted image. In this case, in theknee_function_info SEI, 200 is set as the pre-conversion positioninformation (input_knee_point), and 800 is set as the post-conversionposition information (output_knee_point).

In addition, in the example of FIG. 20, the HDR luminance rangeinformation (d_range) is 4000, and the display luminance rangeinformation (d_range_disp_luminance) is 800 (candela per square meter).

As in the case of FIG. 20, in a case where a coding target image is anHDR image, and a converted image is an SDR image, a knee pointinput_knee_point_PER (%) and luminance output_knee_point_PER (%) of aconverted image corresponding to the knee point are defined by thefollowing Equation (2).

$\begin{matrix}{\lbrack {{Math}.\mspace{14mu} 2} \rbrack\mspace{644mu}} & \; \\{{{{input\_ knee}{\_ point}{\_ DR}} = {\frac{d\_ range}{10} \times \frac{{input\_ knee}{\_ point}}{1000}}}{{{output\_ knee}{\_ point}{\_ DR}} = {100 \times \frac{{output\_ knee}{\_ point}}{1000}}}} & (2)\end{matrix}$

Therefore, the decoding device described later recognizes that, the kneepoint input_knee_point_PER and the luminance output_knee_point_PER are80% according to Equation (2). In addition, the decoding devicedescribed later recognizes that knee conversion corresponding to the DRconversion information is knee compression since the pre-conversionposition information is less than the post-conversion positioninformation. Further, the decoding device described later recognizesthat the maximum value of luminance of the converted image is 400% fromthe HDR luminance range information.

As mentioned above, the decoding device described later knee-compresses80% to 400% of luminance of the SDR image which is obtained as a resultof decoding, to 80% to 100%. Therefore, the decoding device can convertthe HDR image obtained as a result of decoding into a desired SDR image.

<Description of Process in Coding Device>

FIG. 21 is a flowchart illustrating a stream generation processperformed by the coding device 70 of FIG. 16.

In step S71 of FIG. 21, the conversion unit 73 of the coding device 70determines whether or not, for example, a coding target image is an SDRimage in response to an instruction or the like from the user. If it isdetermined that a coding target image is an SDR image in step S71, theprocess proceeds to step S72.

In step S72, the conversion unit 73 converts an HDR image which is inputfrom an external device into an SDR image through knee compression ofluminance of the HDR image, and supplies the SDR image to the codingunit 72.

On the other hand, if it is determined that a coding target image is notan SDR image in step S71, that is, a coding target image is an HDRimage, the conversion unit 73 supplies an HDR image which is input froman external device to the coding unit 72 without change, and the processproceeds to step S73.

In step S73, the setting unit 71 sets an SPS. In step S74, the settingunit 71 sets VUI. In step S75, the setting unit 71 sets a PPS.

In step S76, the setting unit 71 sets knee_function_info SEI in responseto an instruction or the like from a user. The setting unit 71 suppliesthe parameter sets including the set SPS, PPS, VUI, knee_function_infoSEI, and the like to the coding unit 72.

In step S77, the coding unit 72 codes an SDR image or an HDR imagesupplied from the conversion unit 73 as a coding target image in theHEVC method. In step S78, the coding unit 72 generates a coded streamfrom coded data which is obtained as a result of the coding and theparameter sets which are supplied from the setting unit 71, andtransmits the generated coded stream to the transmission unit 13.

In step S79, the transmission unit 13 transmits the coded streamsupplied from the coding unit 72, to the decoding device describedlater, and then finishes the process.

As mentioned above, the coding device 70 sets and transmitsknee_function_info SEI including DR conversion information, and thus thedecoding device described later can convert a coding target imageobtained as a result of decoding into a desired converted image on thebasis of the DR conversion information. Therefore, it can be said thatthe coding device 70 can code an image so that a decoded image can beconverted into a desired converted image during decoding.

In addition, since the DR conversion information is set, the codingdevice 70 can generate a coded stream of an image corresponding to anHDR display and an SDR display only by coding either an SDR image or anHDR image. Therefore, it is possible to further reduce a data amount ofa coded stream than in a case of coding both an HDR image and an SDRimage.

(Configuration Example of Second Embodiment of Decoding Device)

FIG. 22 is a block diagram illustrating a configuration example of asecond embodiment of a decoding device which decodes a coded streamtransmitted from the coding device 70 of FIG. 16 and to which thepresent disclosure is applied.

Among constituent elements illustrated in FIG. 22, the same constituentelements as the constituent elements of FIG. 12 are given the samereference numerals. Repeated description will be omitted as appropriate.

A configuration of a decoding device 90 of FIG. 22 is different from theconfiguration of the decoding device 50 of FIG. 12 in that an extractionunit 91, a decoding unit 92, a conversion unit 93, a display controlunit 94, and a display unit 95 are provided instead of the extractionunit 52, the decoding unit 53, the conversion unit 54, the displaycontrol unit 55, and the display unit 56. The decoding device 90converts a decoded image into a converted image according to the type ofdisplay unit 95, and displays the converted image on the display unit95.

Specifically, the extraction unit 91 of the decoding device 90 extractsparameter sets and coded data from a coded stream which is supplied fromthe reception unit 51. The extraction unit 91 supplies the parametersets and the coded data to the decoding unit 92. In addition, theextraction unit 91 supplies knee_function_info SEI among the parametersets, to the conversion unit 93.

The decoding unit 92 decodes the coded data supplied from the extractionunit 91 in the HEVC method. At this time, the decoding unit 92 alsorefers to the parameter sets supplied from the extraction unit 91 asnecessary. The decoding unit 92 supplies a decoded image to theconversion unit 93.

In a case where a dynamic range of luminance corresponding to thedisplay unit 95 is a dynamic range of luminance of the decoded image,the conversion unit 93 supplies the decoded image which is supplied fromthe decoding unit 92, to the display control unit 94 without change. Onthe other hand, in a case where a dynamic range of luminancecorresponding to the display unit 95 is not a dynamic range of luminanceof the decoded image, the conversion unit 93 converts the decoded imageinto an converted image through knee conversion on the basis of DRconversion information included in the knee_function_info SEI suppliedfrom the extraction unit 91. In addition, the conversion unit 93supplies the converted image to the display control unit 94 as a displayimage.

Specifically, in a case where the display unit 95 is an HDR display, andthe decoded image is an HDR image, or in a case where the display unit95 is an SDR display, and the decoded image is an SDR image, theconversion unit 93 supplies the decoded image to the display controlunit 94 without change. On the other hand, in a case where the displayunit 95 is an SDR display, and the decoded image is an HDR image, or ina case where the display unit 95 is an HDR display, and the decodedimage is an SDR image, the conversion unit 93 performs knee conversionon the decoded image on the basis of the DR conversion information so asto generate a converted image. In addition, the conversion unit 93supplies the converted image to the display control unit 94 as a displayimage.

The display control unit 94 displays the display image supplied from theconversion unit 93 on the display unit 95. Accordingly, in a case wherethe display unit 95 is an HDR display, an HDR image is displayed on thedisplay unit 95, and in a case where the display unit 95 is an SDRdisplay, an SDR image is displayed on the display unit 95. The displayunit 95 is an HDR display or an SDR display, and displays a displayimage supplied from the display control unit 94.

<Description of Process in Decoding Device>

FIG. 23 is a flowchart illustrating an image generation processperformed by the decoding device 90 of FIG. 22.

In step S91 of FIG. 23, the reception unit 51 of the decoding device 90receives a coded stream transmitted from the coding device 70 of FIG.16, and supplies the coded stream to the extraction unit 91.

In step S92, the extraction unit 91 extracts parameter sets and codeddata from the coded stream which is supplied from the reception unit 51.The extraction unit 91 supplies the parameter sets and the coded data tothe decoding unit 92. In addition, the extraction unit 91 suppliesknee_function_info SEI among the parameter sets, to the conversion unit93.

In step S93, the decoding unit 92 decodes the coded data supplied fromthe extraction unit 91 in the HEVC method. At this time, the decodingunit 92 also refers to the parameter sets supplied from the extractionunit 91 as necessary. The decoding unit 92 supplies a decoded image tothe conversion unit 93.

In step S94, the conversion unit 93 acquires DR conversion informationfrom the knee_function_info SEI which is supplied from the extractionunit 91.

In step S95, the conversion unit 93 determines whether or not a dynamicrange of luminance corresponding to the display unit 95 is a dynamicrange of luminance of the decoded image. If it is determined that adynamic range of luminance corresponding to the display unit 95 is not adynamic range of luminance of the decoded image, the process proceeds tostep S96.

In step S96, the conversion unit 93 converts the decoded image suppliedfrom the decoding unit 92 into a converted image on the basis of the DRconversion information, and supplies the converted image to the displaycontrol unit 94 as a display image. In addition, the process proceeds tostep S97.

On the other hand, it is determined in step S95 that a dynamic range ofluminance corresponding to the display unit 95 is a dynamic range ofluminance of the decoded image, the conversion unit 93 supplies thedecoded image which is supplied from the decoding unit 92, to thedisplay control unit 94 as a display image without change. In addition,the process proceeds to step S97.

In step S97, the display control unit 94 displays the display imagesupplied from the conversion unit 93 on the display unit 95, andfinishes the process.

As mentioned above, the decoding device 90 converts the decoded imageinto the converted image on the basis of the DR conversion information,and thus can convert a decoded image to a desired converted image.

In addition, in the second embodiment, one of an SDR image and an HDRimage is a coding target image, and the other is a converted image, butan SDR image may be replaced with a desensitized development image of anHDR image in which an expected value of brightness of the display unitcorresponding to the maximum value of luminance is greater than that ofthe SDR image.

(Second Example of DR Conversion Information)

FIGS. 24 and 25 are diagrams illustrating examples of DR conversioninformation set in knee_function_info SEI in a case where one of adesensitized development image and an HDR image is a coding target imageand the other is a converted image.

In addition, in the examples of FIGS. 24 and 25, the desensitizeddevelopment image is an image in which a dynamic range of luminance is 0to 200%, obtained by performing desensitized development of 1 EV(exposure value) on an HDR image. Further, an expected value ofbrightness of the display unit corresponding to the maximum value ofluminance of the desensitized development image is 400 (candela persquare meter) higher than 200 (candela per square meter) which is anexpected value of brightness corresponding to the maximum value ofluminance in an SDR image.

Information indicating that a coding target image or a converted imageis an image obtained by performing desensitized development on an HDRimage, and a dynamic range of luminance of the desensitized developmentimage are set in tone_mapping_info_SEI by the setting unit 71.

In the example of FIG. 24, a coding target image is a desensitizeddevelopment image, and the user sets an HDR image which is obtained as aresult of knee-decompressing 160% to 200% of luminance of thedesensitized development image to 160% to 400%, as a desired convertedimage. In this case, in the knee_function_info SEI, 800 is set as thepre-conversion position information (input_knee_point), and 400 is setas the post-conversion position information (output_knee_point).

In addition, in the example of FIG. 24, the HDR luminance rangeinformation (d_range) is 4000, and the display luminance range(d_range_disp_luminance) is 800 (candela per square meter).

As in the case of FIG. 24, in a case where a coding target image is adesensitized development image, and a converted image is an HDR image, aknee point input_knee_point_PER (%) and luminance output knee_point_PER(%) of a converted image corresponding to the knee point are defined bythe above Equation (1).

Therefore, the decoding device 90 recognizes that the knee pointinput_knee_point_PER and the luminance output_knee_point_PER are 160%according to Equation (1). In addition, the decoding device 90recognizes that the maximum value of luminance of the converted image is400% from the HDR luminance range information. Further, the decodingdevice 90 recognizes that a dynamic range of luminance of the codingtarget image is 0 to 200% from the tone_mapping_info_SEI. Furthermore,in a case where the display unit 95 is an HDR display, 160% to 200% ofluminance of a desensitized development image which is obtained as aresult of decoding is knee-decompressed to 160% to 400% so as to bedisplayed as a display image.

On the other hand, in a case where the display unit 95 is an SDRdisplay, the decoding device 90 displays a desensitized developmentimage as a display image without change. At this time, an expected valueof brightness of the display unit corresponding to the maximum value ofluminance of the desensitized development image is greater than that ofan SDR image, and thus brightness of the display image is insufficient.

However, recently, an SDR display (hereinafter, referred to as a highluminance SDR display) of which brightness corresponding to the maximumvalue of luminance is relatively high 300 (candela per square meter) orthe like has been developed. In a case where the display unit 95 is ahigh luminance SDR display, brightness of a display image can besufficiently maintained even if a desensitized development image isdisplayed as the display image without change. In addition, since acompression ratio of knee compression during generation of a codingtarget image is lower than in a case where a coding target image is anSDR image, quality of a display image can be improved.

In the example of FIG. 25, a coding target image is an HDR image, andthe user sets a desensitized development image which is obtained as aresult of knee-compressing 160% to 400% of luminance of the HDR image to160% to 200%, as a desired converted image. In this case, in theknee_function_info SEI, 400 is set as the pre-conversion positioninformation (input_knee_point), and 800 is set as the post-conversionposition information (output_knee_point).

In addition, in the example of FIG. 25, the HDR luminance rangeinformation (d_range) is 4000, and the display luminance range(d_range_disp_luminance) is 800 (candela per square meter).

As in the case of FIG. 25, in a case where a coding target image is anHDR image, and a converted image is a desensitized development image, aknee point input_knee_point_PER (%) and luminance output_knee_point_PER(%) of a converted image corresponding to the knee point are defined bythe above Equation (2).

Therefore, the decoding device 90 recognizes that the knee pointinput_knee_point_PER and the luminance output_knee_point_PER are 160%according to Equation (2). In addition, the decoding device 90recognizes that the maximum value of luminance of the coding targetimage is 400% from the HDR luminance range information. Further, thedecoding device 90 recognizes that a dynamic range of luminance of theconverted image is 0 to 200% from the tone_mapping_info_SEI.

Furthermore, in a case where the display unit 95 is an SDR display, thedecoding device 90 knee-compresses 160% to 400% of luminance of an HDRimage which is obtained as a result of decoding to 160% to 200% so as todisplay a compressed result as a display image. In this case, asdescribed above, brightness of the display image is insufficient.However, in a case where the display unit 95 is a high luminance SDRdisplay, brightness of a display image can be sufficiently maintained asdescribed above. In addition, quality of a display image can beimproved.

On the other hand, in a case where the display unit 95 is an HDRdisplay, the decoding device 90 displays an HDR image which is obtainedas a result of decoding as a display image without change.

In addition, the DR conversion information of FIG. 17 may be included inSEI such as tone_mapping_info_SEI other than knee_function_info SEI.

(First Example of Syntax of Tone_Mapping_Info_SEI)

FIG. 26 is a diagram illustrating an example of syntax oftone_mapping_info_SEI in a case where the DR conversion information ofFIG. 17 is included in the tone_mapping_info SEI.

The tone_mapping_info_SEI is SEI regarding conversion of luminance. Asillustrated in FIG. 26, in a case where the DR conversion information ofFIG. 17 is included in the tone_mapping_info_SEI, tone_map_model_idindicating a conversion model of luminance is set to, for example, 5. Inaddition, in the tone_mapping_info_SEI, pre-conversion positioninformation (input_knee_point), post-conversion position information(output_knee_point), HDR luminance range information (d_range), anddisplay luminance information (d_range_disp_luminance) are set in thetone_mapping_info_SEI as DR conversion information.

In addition, the HDR luminance range information (d_range) and thedisplay luminance information (d_range_disp_luminance) are included intone_mapping_info_SEI when tone_map_model_id is 4. Therefore, asillustrated in FIG. 27, the HDR luminance range information (d_range)and the display luminance information (d_range_disp_luminance) may notbe included in the tone_mapping_info_SEI. Further, only one of the HDRluminance range information (d_range) and the display luminanceinformation (d_range_disp_luminance) may be included.

(Second Example of Syntax of Knee_Function_Info SEI)

FIG. 28 is a diagram illustrating a second example of syntax ofknee_function_info SEI set by the setting unit 71 of FIG. 16, and FIG.29 is a diagram illustrating each piece of information set in theknee_function_info SEI of FIG. 28.

A plurality of knee points are set in the knee_function_info SEI of FIG.28. Specifically, in the same manner as in the case of FIG. 17, a kneeconversion ID (knee_function_id) and a knee conversion cancel flag(knee_function_cancel_flag) are set in the knee_function_info SEI ofFIG. 28.

In addition, if the knee conversion cancel flag is 0, as illustrated inFIG. 28, DR conversion information is set in the knee_function_info SEI.The DR conversion information is the same as in the case of FIG. 17except that a compression flag (compression_flag) and a knee pointnumber (num_knee_point_minus1) are included, and pre-conversion positioninformation (input_knee_point) and post-conversion position information(output_knee_point) are set for each knee point. Description of the samepart as in the case of FIG. 17 is repeated and thus will be omitted asappropriate.

As illustrated in FIG. 29, the compression flag is a flag indicatingwhether or not knee conversion is knee compression. In other words, in acase where the number of knee points is one, when the pre-conversionposition information (input_knee_point) is equal to or more than thatthe post-conversion position information (output_knee_point), it can bedetermined that knee conversion is knee decompression, and when thepre-conversion position information (input_knee_point) is less than thepost-conversion position information (output_knee_point), it can bedetermined that knee conversion is knee compression.

However, in a case where there are a number of knee points, it is unableto be accurately determined whether knee conversion is kneedecompression or knee compression by using the magnitude correlationbetween the pre-conversion position information and the post-conversionposition information, and thus the compression flag is set. In addition,even in a case where the number of knee points is one, the compressionflag may be set.

The compression flag is set to 1 when knee conversion is kneecompression, and is set to 0 when knee conversion is knee decompression.

The knee point number is a value obtained by subtracting 1 from thenumber of knee points. In addition, an order i (where is an integer of 0or more) in which pre-conversion position information andpost-conversion position information of knee points are set is an orderin which the pre-conversion position information is reduced.

(Third Example of DR Conversion Information)

FIGS. 30 and 31 are diagrams illustrating examples of DR conversioninformation set in the knee_function_info SEI of FIG. 28.

In the example of FIG. 30, a coding target image is an SDR image. Inaddition, the user sets an HDR image which is obtained as a result ofrespectively converting 0 to 60%, 60% to 80%, 80% to 90%, and 90% to100% of an SDR image into 0 to 40%, 40% to 100%, 100% to 180%, and 180%to 400%, as a desired converted image.

In this case, in the knee_function_info SEI, 600 is set aspre-conversion position information (input_knee_point[0]) of the 0-thknee point, and 100 is set as post-conversion position information(output_knee_point[0]) thereof. 800 is set as pre-conversion positioninformation (input_knee_point[1]) of the first knee point, and 250 isset as post-conversion position information (output_knee_point[1])thereof. 900 is set as pre-conversion position information(input_knee_point[2]) of the second knee point, and 450 is set aspost-conversion position information (output_knee_point[2]) thereof.

In addition, in the example of FIG. 30, the HDR luminance rangeinformation (d_range) is 4000, the display luminance range(d_range_disp_luminance) is 800 (candela per square meter), and thecompression flag (compression_flag) is 0.

As described above, in a case where a coding target image is an SDRimage, and a converted image is an HDR image, a knee pointinput_knee_point_PER (%) and luminance output_knee_point_PER (%) of aconverted image corresponding to the knee point are defined by the aboveEquation (1).

Therefore, the decoding device 90 recognizes that the 0-th to secondknee points input_knee_point_PER are respectively 60%, 80%, and 90%according to Equation (1). In addition, the decoding device 90recognizes that the 0-th to second luminances output_knee_point_PER arerespectively 40%, 100%, and 180%. Further, the decoding device 90recognizes that the maximum value of luminance of the converted image is400% from the HDR luminance range information.

Furthermore, the decoding device 90 respectively knee-converts 0 to 60%,60% to 80%, 80% to 90%, and 90% to 100% of an SDR image which isobtained as a result of decoding into 0 to 40%, 40% to 100%, 100% to180%, and 180% to 400%, according to a conversion straight line in whichthe knee points are connected to each other in a set order. Therefore,the decoding device 90 can convert the SDR image which is obtained as aresult of decoding, into a desired HDR image.

In the example of FIG. 31, a coding target image is an HDR image. Inaddition, the user sets an SDR image which is obtained as a result ofrespectively converting 0 to 40%, 40% to 100%, 100% to 180%, and 180% to400% of luminance of an HDR image into 0 to 60%, 60% to 80%, 80% to 90%,and 90% to 100%, as a desired converted image.

In this case, in the knee_function_info SEI, 100 is set aspre-conversion position information (input_knee_point[0]) of the 0-thknee point, and 600 is set as post-conversion position information(output_knee_point[0]). 250 is set as pre-conversion positioninformation (input_knee_point[1]) of the first knee point, and 800 isset as post-conversion position information (output_knee_point[1]). 450is set as pre-conversion position information (input_knee_point[2]) ofthe second knee point, and 900 is set as post-conversion positioninformation (output_knee_point[2]).

In addition, in the example of FIG. 31, the HDR luminance rangeinformation (d_range) is 4000, the display luminance range(d_range_disp_luminance) is 800 (candela per square meter), and thecompression flag (compression_flag) is 1.

As described above, in a case where a coding target image is an HDRimage, and a converted image is an SDR image, a knee pointinput_knee_point_PER (%) and luminance output_knee_point_PER (%) of aconverted image corresponding to the knee point are defined by the aboveEquation (2).

Therefore, the decoding device 90 recognizes that the 0-th to secondknee points input_knee_point_PER are respectively 40%, 100%, and 180%according to Equation (2). In addition, the 0-th to second luminancesoutput_knee_point_PER (%) are respectively 60%, 80%, and 90%. Inaddition, the decoding device 90 recognizes that the maximum value ofluminance of the converted image is 400% from the HDR luminance rangeinformation.

Further, the decoding device 90 knee-converts 0 to 40%, 40% to 100%,100% to 180%, and 180% to 400% of an HDR image which is obtained as aresult of decoding into 0 to 60%, 60% to 80%, 80% to 90%, and 90% to100% by connecting the knee points to each other in a set order.Therefore, the decoding device 90 can convert the HDR image which isobtained as a result of decoding, into a desired SDR image.

As mentioned above, in a case where a plurality of knee points are set,a compression ratio can be more finely set than in a case where a singleknee point is set. Therefore, it is possible to perform knee conversionwith higher accuracy.

In addition, the DR conversion information of FIG. 28 may be included inSEI such as tone_mapping_info_SEI other than knee_function_info SEI.

(Second Example of Syntax of Tone_Mapping_Info_SE)

FIG. 32 is a diagram illustrating an example of syntax oftone_mapping_info_SEI in a case where the DR conversion information ofFIG. 28 is included in the tone_mapping_info_SEI.

As illustrated in FIG. 32, in a case where the DR conversion informationof FIG. 28 is included in the tone_mapping_info_SEI, tone_map_model_idis set to, for example, 5. In addition, in the tone_mapping_info_SEI,compression flag (compression_flag (mapping_flag)), HDR luminance rangeinformation (d_range), display luminance information(d_range_disp_luminance), a knee point number (num_knee_point_minus1),and pre-conversion position information (input_knee_point) andpost-conversion position information (output_knee_point) of each kneepoint are set in the tone_mapping_info_SEI as DR conversion information.

In addition, in the same manner as in the tone_mapping_info_SEI of FIG.27, the HDR luminance range information (d_range) and the displayluminance information (d_range_disp_luminance) may not be included inthe tone_mapping_info_SEI of FIG. 32. Further, only one of the HDRluminance range information (d_range) and the display luminanceinformation (d_range_disp_luminance) may be included.

Furthermore, the knee point number (num_knee_point_minus1) may be anyone of 0, 1, and 2 as illustrated in FIGS. 33 to 35. In other words, theknee point number (num_knee_point_minus1) may be limited to 2 or less.In this case, as illustrated in FIGS. 33 to 35, the number of bits ofthe knee point number (num_knee_point_minus1) included in theknee_function_info SEI or the tone_mapping_info_SEI is fixed to 2 bits(u(2)). As mentioned above, the maximum value of the knee point number(num_knee_point_minus1) is determined, and thus an amount of DRconversion information can be reduced. Accordingly, the DR conversioninformation can be transmitted with a small packet as in AVI InfoFrameof High-Definition Multimedia Interface (HDMI (registered trademark)).

(Third Example of Syntax of Knee_Function_Info_SEI)

FIG. 36 is a diagram illustrating a third example of syntax ofknee_function_info SEI set by the setting unit 71 of FIG. 16, and FIG.37 is a diagram illustrating each piece of information set in theknee_function_info SEI of FIG. 36.

A plurality of knee points and a knee point (hereinafter, referred to asa representative knee point) which is representatively used are set inthe knee_function_info SEI of FIG. 36.

Specifically, in the same manner as in the case of FIG. 17, a kneeconversion ID (knee_function_id) and a knee conversion cancel flag(knee_function_cancel_flag) are set in the knee_function_info SEI ofFIG. 36.

In addition, if the knee conversion cancel flag is 0, as illustrated inFIG. 36, DR conversion information is set in the knee_function_info SEI.The DR conversion information is the same as in the case of FIG. 28except that representative pre-conversion position information(representative_input_knee_point) and representative post-conversionposition information (representative_output_knee_point) are included.Description of the same part as in the case of FIG. 28 is repeated andthus will be omitted as appropriate.

As illustrated in FIG. 37, the representative pre-conversion positioninformation is information indicating a representative knee point of acoding target image which is an unconverted image in conversioncorresponding to the DR conversion information, and is a permillage ofthe representative knee point when the maximum value of luminance of thecoding target image is set to 1000 permil.

The representative pre-conversion position information is informationindicating luminance corresponding to a representative knee point of aconverted image in conversion corresponding to the DR conversioninformation, and is a permillage of luminance corresponding to a kneepoint when the maximum value of luminance of the converted image is setto 1000 permil.

In addition, the representative knee point may be one of knee pointscorresponding to a plurality of pre-conversion position informationpieces included in the DR conversion information, and may be a kneepoint which is completely different from the knee point.

(Fourth Example of DR Conversion Information)

FIG. 38 is a diagram illustrating an example of DR conversioninformation set in the knee_function_info SEI of FIG. 36.

In the example of FIG. 38, a coding target image is an SDR image. Inaddition, the user sets an HDR image which is obtained as a result ofrespectively converting 0 to 60%, 60% to 80%, 80% to 90%, and 90% to100% of an SDR image into 0 to 40%, 40% to 100%, 100% to 180%, and 180%to 400%, as a desired converted image when the decoding device 90performs the knee conversion with high accuracy. Further, the user setsan HDR image which is obtained by knee-decompressing 80% to 100% ofluminance of an SDR image to 80% to 400%, as a desired converted imagewhen the decoding device 90 performs simple knee conversion with lowaccuracy.

In this case, in the knee_function_info SEI, the same values as in FIG.30 are set as pre-conversion position information (input_knee_point) andpost-conversion position information (output_knee_point) of the 0-th tosecond knee points. In addition, the representative pre-conversionposition information (representative_input_knee_point) is 800, and therepresentative post-conversion position information(representative_output_knee_point) is 200.

In addition, in the example of FIG. 38, the HDR luminance rangeinformation (d_range) is 4000, the display luminance range(d_range_disp_luminance) is 800 (candela per square meter), and thecompression flag (compression_flag) is 0.

As illustrated in FIG. 38, in a case where the decoding device 90performs simple knee conversion with low accuracy, the decoding device90 recognizes that the representative knee pointrepresentative_input_knee_point_PER (%) and luminancerepresentative_output_knee_point_PER (%) of a converted imagecorresponding to the representative knee point are 80% according to theabove Equation (1). In addition, the decoding device 90 recognizes thatthe maximum value of luminance of the converted image is 400% from theHDR luminance range information. Further, the decoding device 90knee-decompresses 80% to 100% of luminance of an SDR image which isobtained as a result of decoding, to 80% to 400%. Therefore, thedecoding device 90 can convert the SDR image which is obtained as aresult of decoding into a desired HDR image.

On the other hand, in a case where the decoding device 90 performs kneeconversion with high accuracy, the decoding device 90 performs the sameprocess as in FIG. 30, and converts an SDR image which is obtained as aresult of decoding into a desired HDR image.

As mentioned above, the representative pre-conversion positioninformation (representative_input_knee_point) and the representativepost-conversion position information (representative_output_knee_point)are included in the DR conversion information of FIG. 36. Therefore,even in a case where a processing rate or a resource such as a memorycapacity is unable to be sufficiently secured in the decoding device 90,knee conversion can be performed on the basis of a representative kneepoint. In addition, since the representative pre-conversion positioninformation and the representative post-conversion position informationare transmitted to the decoding device 90, the decoding device 90 doesnot have to generate representative pre-conversion position informationand representative post-conversion position information on the basis ofpre-conversion position information and post-conversion positioninformation of a plurality of knee points.

In addition, the DR conversion information of FIG. 36 may be included inSEI such as tone_mapping_info_SEI other than knee_function_info SEI.

(Third Example of Syntax of Tone_Mapping_Info_SEI)

FIG. 39 is a diagram illustrating an example of syntax oftone_mapping_info_SEI in a case where the DR conversion information ofFIG. 36 is included in the tone_mapping_info_SEI.

As illustrated in FIG. 39, in a case where the DR conversion informationof FIG. 36 is included in the tone_mapping_info_SEI, tone_map_model_idis set to, for example, 5. In addition, in the tone_mapping_info_SEI, acompression flag (compression_flag), representative pre-conversionposition information (representative_input_knee_point), representativepost-conversion position information (representative_output_knee_point),HDR luminance range information (d_range), display luminance information(d_range_disp_luminance), a knee point number (num_knee_point_minus1),and pre-conversion position information (input_knee_point) andpost-conversion position information (output_knee_point) of each kneepoint are set in the tone_mapping_info_SEI as DR conversion information.

In addition, in the same manner as in the tone_mapping_info_SEI of FIG.27, the HDR luminance range information (d_range) and the displayluminance information (d_range_disp_luminance) may not be included inthe tone_mapping_info_SEI of FIG. 39. Further, only one of the HDRluminance range information (d_range) and the display luminanceinformation (d_range_disp_luminance) may be included.

(Fourth Example of Syntax of Knee_Function_Info_SEI)

FIG. 40 is a diagram illustrating a fourth example of syntax ofknee_function_info SEI set by the setting unit 71 of FIG. 16, and FIG.41 is a diagram illustrating (semantics of) each piece of informationset in the knee_function_info SEI of FIG. 40.

In the knee_function_info SEI of FIG. 40, images other than an SDR imagecan be employed as one of a coding target image and a converted image.

Specifically, in the same manner as in the case of FIG. 17, a kneeconversion ID (knee_function_id) and a knee conversion cancel flag(knee_function_cancel_flag) are set in the knee_function_info SEI ofFIG. 40.

In addition, if the knee conversion cancel flag is 0, as illustrated inFIG. 40, DR conversion information is set in the knee_function_info SEI.The DR conversion information is the same as in the case of FIG. 28except that knee conversion persistence flag(knee_function_persistence_flag) is newly included, and unconverteddisplay range information (input_d_range), unconverted display luminanceinformation (input_disp_luminance), converted display range information(output_d_range), and converted display luminance information(output_disp_luminance) are included instead of the HDR luminance rangeinformation (d_range) and the display luminance information(d_range_disp_luminance). Description of the same part as in the case ofFIG. 28 is repeated and thus will be omitted as appropriate.

As illustrated in FIG. 41, the knee conversion persistence flag is aflag indicating whether or not the DR conversion information is appliedto a plurality of pictures which are continuously located. The kneeconversion persistence flag is set to 1 when the DR conversioninformation is applied to a plurality of pictures which are continuouslylocated, and is set to 0 when the DR conversion information is appliedto only one picture. The knee conversion persistence flag may also beset in the knee_function_info SEI of FIGS. 17, 28, 34 and 36.

In addition, the unconverted luminance range information is informationindicating a permillage of the maximum value of luminance of a codingtarget image which is an unconverted image in conversion correspondingto the DR conversion information, and the converted luminance rangeinformation is information indicating a permillage of the maximum valueof luminance of a converted image.

In addition, the unconverted display luminance information isinformation indicating an expected value of brightness of the displayunit corresponding to the maximum value of luminance of a coding targetimage, and the converted display luminance information is informationindicating an expected value of brightness of the display unitcorresponding to the maximum value of luminance of a converted image.

(Fifth Example of DR Conversion Information)

FIGS. 42 and 43 are diagrams illustrating examples of DR conversioninformation set in the knee_function_info SEI of FIG. 40.

In the example of FIG. 42, a coding target image is an HDR image(hereinafter, referred to as a 200% HDR image) whose dynamic range is 0to 200%. In addition, the user sets a 400% HDR image which is obtainedas a result of respectively knee-converting 0 to 120%, 120% to 160%,160% to 180%, and 180% to 200% of luminance of a 200% HDR image into 0to 40%, 40% to 100%, 100% to 180%, and 180% to 400%, as a desiredconverted image. The 400% HDR image is an HDR image whose dynamic rangeis 0 to 400%.

In this case, in the knee_function_info SEI, the same values as in FIG.30 are set as pre-conversion position information (input_knee_point) andpost-conversion position information (output_knee_point) of the 0-th tosecond knee points. In addition, 2000 is set as the unconvertedluminance range information (input_d_range), and 4000 is set as theconverted luminance range information (output_d_range).

Further, in the example of FIG. 42, the unconverted display luminanceinformation (input_disp_luminance) is 400 (candela per square meter),and the converted display luminance information (output_disp_luminance)is 800 (candela per square meter). The compression flag(compression_flag) is 0.

As illustrated in FIG. 42, in a case where a coding target image is animage with a dynamic range corresponding to the unconverted luminancerange information, and a converted image is an image with a dynamicrange corresponding to the converted luminance range information, a kneepoint input_knee_point_PER (%) and luminance output_knee_point_PER (%)of a converted image corresponding to the knee point are defined by thefollowing Equation (3).

$\begin{matrix}{\lbrack {{Math}.\mspace{14mu} 3} \rbrack\mspace{644mu}} & \; \\{{{{input\_ knee}{\_ point}{\_ DR}} = {\frac{{input\_ d}{\_ range}}{10} \times \frac{{input\_ knee}{\_ point}}{1000}}}{{{output\_ knee}{\_ point}{\_ DR}} = {\frac{{output\_ d}{\_ range}}{10} \times \frac{{output\_ knee}{\_ point}}{1000}}}} & (3)\end{matrix}$

Therefore, the decoding device 90 recognizes that the 0-th to secondknee points input_knee_point_PER are respectively 120%, 160%, and 180%according to Equation (3). In addition, the decoding device 90recognizes that the 0-th to second luminances output_knee_point_PER arerespectively 40%, 100%, and 180%. Further, the decoding device 90recognizes that the maximum value of luminance of the coding targetimage is 200% from the input luminance range information, and themaximum value of luminance of the converted image is 400% from theoutput luminance range information.

Furthermore, the decoding device 90 respectively knee-converts 0 to120%, 120% to 160%, 160% to 180%, and 180% to 200% of a 200% HDR imagewhich is obtained as a result of decoding into 0 to 40%, 40% to 100%,100% to 180%, and 180% to 400%, according to a conversion straight linein which the knee points are connected to each other in a set order.Therefore, the decoding device 90 can convert the 200% HDR image whichis obtained as a result of decoding, into a desired 400% HDR image.

In the example of FIG. 43, a coding target image is a 400% HDR image. Inaddition, the user sets a 200% HDR image which is obtained as a resultof respectively knee-converting0 to 40%, 40% to 100%, 100% to 180%, and180% to 400% of luminance of a 400% HDR image, into 0 to 120%, 120% to160%, 160% to 180%, and 180% to 200% as a desired converted image.

In this case, in the knee_function_info SEI, the same values as in FIG.31 are set as pre-conversion position information (input_knee_point) andpost-conversion position information (output_knee_point) of the 0-th tosecond knee points. In addition, 4000 is set as the unconvertedluminance range information (input_d_range), and 2000 is set as theconverted luminance range information (output_d_range).

Further, in the example of FIG. 43, the unconverted display luminanceinformation (input_disp_luminance) is 800 (candela per square meter),and the converted display luminance information (output_disp_luminance)is 400 (candela per square meter). The compression flag(compression_flag) is 1.

As described above, in a case where a coding target image is an imagewith a dynamic range corresponding to the unconverted luminance rangeinformation, and a converted image is an image with a dynamic rangecorresponding to the converted luminance range information, a knee pointinput_knee_point_PER (%) and luminance output_knee_point_PER (%) of aconverted image corresponding to the knee point are defined by the aboveEquation (3).

Therefore, the decoding device 90 recognizes that the 0-th to secondknee points input_knee_point_PER are respectively 40%, 100%, and 180%according to Equation (3). In addition, the decoding device 90recognizes that the 0-th to second luminances output_knee_point_PER (%)are respectively 120%, 160%, and 180%. Further, the decoding device 90recognizes that the maximum value of luminance of the coding targetimage is 400% from the input luminance range information, and themaximum value of luminance of the converted image is 200% from theoutput luminance range information.

Furthermore, the decoding device 90 respectively knee-converts 0 to 40%,40% to 100%, 100% to 180%, and 180% to 400% of a 400% HDR image which isobtained as a result of decoding into 0 to 120%, 120% to 160%, 1.60% to180%, and 180% to 200%, by connecting the knee points to each other in aset order. Therefore, the decoding device 90 can convert the 400% HDRimage which is obtained as a result of decoding, into a desired 200% HDRimage.

As mentioned above, according to the DR conversion information of FIG.40, not only conversion between an SDR image and an HDR image but alsoconversion between HDR images with different dynamic ranges can beperformed as desired by a user in the decoding device 90. A dynamicrange of an HDR image may be greater than 0 to 100%, and may be 0 to400%, 0 to 800%, 0 to 1300%, and the like. In addition, an expectedvalue of brightness of the display corresponding to the maximum value ofluminance of an HDR image may be greater than 100 (candela per squaremeter), and may be 800 (candela per square meter), 4000 (candela persquare meter), 1500 (candela per square meter), and the like.

[Description of Operation of Decoding Device]

FIG. 44 is a diagram illustrating an operation of the decoding device 90in a case where the knee_function_info SEI of FIG. 40 is set in aplurality.

In an example of FIG. 44, a coding target image is a 400% HDR image. Inaddition, knee_function_info SEI (hereinafter, referred to as 800% HDRimage knee_function_info SEI) for setting a desired converted image to a800% HDR image whose dynamic range is 0 to 800%, and knee_function_infoSEI (hereinafter, referred to as SDR image knee_function_info SEI) forsetting a desired converted image to an SDR image, are set. In thiscase, different knee conversion IDs are given to the 800% HDR imageknee_function_info SEI and the SDR image knee_function_info SEI.

In a case where the display unit 95 is an HDR display which can displaya 800% HDR image, the decoding device 90 knee-decompresses luminance ofa 400% HDR image which is a decoded image on the basis of the 800% HDRimage knee_function_info SEI, so as to generate a desired 800% HDR imageas a display image.

On the other hand, in a case where the display unit 95 is an HDR displaywhich can display a 400% HDR image, the decoding device 90 uses a 400%HDR image which is a decoded image as a display image without change. Inaddition, in a case where the display unit 95 is an SDR display, thedecoding device 90 knee-compresses luminance of a 400% HDR image whichis a decoded image on the basis of the SDR image knee_function_info SEI,so as to generate a desired SDR image as a display image.

In addition, the DR conversion information of FIG. 40 may be included inSEI such as tone_mapping_info_SEI other than knee_function_info SEI.

(Fourth Example of Syntax of Tone_Mapping_Info_SEI)

FIG. 45 is a diagram illustrating an example of syntax oftone_mapping_info_SEI in a case where the DR conversion information ofFIG. 40 is included in the tone_mapping_info_SEI.

As illustrated in FIG. 45, in a case where the DR conversion informationof FIG. 40 is included in the tone_mapping_info_SEI, tone_map_model_idis set to, for example, 5. In addition, in the tone_mapping_info_SEI, acompression flag (compression_flag), input luminance range information(input_d_range), input display luminance range(input_d_range_disp_luminance), output luminance range information(output_d_range), output display luminance range(output_d_range_disp_luminance), a knee point number(num_knee_point_minus1), and pre-conversion position information(input_knee_point) and post-conversion position information(output_knee_point) of each knee point are set in thetone_mapping_info_SEI as DR conversion information.

In addition, at least one of the input luminance range information(input_d_range), the input display luminance range(input_d_range_disp_luminance), the output luminance range information(output_d_range), and the output display luminance range(output_d_range_disp_luminance) may not be included in thetone_mapping_info_SEI of FIG. 45.

In addition, in the above description, the DR conversion information isdisposed in SEI, but may be disposed in a system layer.

(Example of Disposing DR Conversion Information in Box of MP4)

[Description of Box of MP4 in which DR Conversion Information isDisposed]

FIG. 46 is a diagram illustrating a box of MP4 as a system layer inwhich DR conversion information is disposed.

As illustrated in FIG. 46, in a case where DR conversion information isdisposed in a box of MP4, a tinf (Tone Mapping Information Box) boxwhich stores DR conversion information as ToneMapInfo is newly defined.The tinf box is stored in a trak box (track box) (a stbl box storedtherein) or a traf box (track fragment box).

(Example of Syntax of ToneMapInfo)

FIG. 47 is a diagram illustrating an example of syntax of ToneMapInfo.

ToneMapInfo of FIG. 47 has the same configuration as that of thetone_mapping_info_SEI of FIG. 32 except that padding_value for bytealignment is inserted thereinto.

In addition, although not illustrated, ToneMapInfo may have the sameconfiguration as that of the tone_mapping_info_SEI of FIG. 26, 27, 39,or 45 except that padding_value for byte alignment is insertedthereinto.

In addition, in the same manner as in the second embodiment, theconversion information in the first embodiment may be disposed in asystem layer.

In addition, an HDR image desired by a user may be an HDR image which isinput to the coding device 70.

Further, in the second embodiment, an HDR image is input to the codingdevice 70, but an SDR image may be input thereto. In this case, when acoding target image is an HDR image, the coding device 70 converts anSDR image which is input from an external device into an HDR image whichis then set as a coding target image.

In addition, a plurality of knee points are set in theknee_function_info SEI of FIG. 40. Therefore, knee conversion of asmoother and more complex function can be defined than in a case whereonly one knee point is set. As a result, the conversion unit 93 canperform the optimal knee conversion.

However, if the number of knee points increases, an amount of DRconversion information increases. Therefore, for example, in a casewhere a decoded image and DR conversion information are transmitted withHDMI, an amount of the DR conversion information is equal to or largerthan 27 bytes which is a size of one packet of AVI InfoFrame of HDMI,and thus the DR conversion information may not be included in AVIInfoFrame.

Therefore, in a third embodiment described later, a decoding deviceperforms thinning-out of an optimal knee point in a case where an amountof DR conversion information is reduced, such as a case where the DRconversion information is transmitted with HDMI.

[Third Embodiment]

(First Example of Semantics)

A first configuration of a third embodiment of a coding device to whichthe present disclosure is applied is the same as the configuration ofthe coding device 70 of FIG. 16 except for the order i of knee pointsand semantics indicated by the knee_function_info SEI of FIG. 40 set bythe setting unit 71. Therefore, hereinafter, only the order i of kneepoints and semantics indicated by the knee_function_info SEI of FIG. 40will be described.

In the first configuration of the third embodiment of the coding deviceto which the present disclosure is applied, the order i of knee pointsis set in an order in which priorities for representing a desiredfunction of knee conversion are higher in the knee_function_info SEI ofFIG. 40.

In addition, FIG. 48 is a diagram illustrating that semantics in thefirst configuration of the third embodiment of the coding device towhich the present disclosure is applied is different from that in thesecond embodiment.

As illustrated in FIG. 48, in the semantics of FIG. 40 in the firstconfiguration of the third embodiment of the coding device to which thepresent disclosure is applied, pre-conversion position information(input_knee_point[i]) of an i-th knee point may be equal to or less thanpre-conversion position information (input_knee_point[i−1]) of a(i−1)-th knee point. In other words, the order i (where i is an integerof 0 or more) in which the pre-conversion position information and thepost-conversion position information of a knee point are set may not bean order in which the post-conversion position information is less.

In addition, a function (knee function) of knee conversion is a straightline which connects knee points to each other in an order (ascendingorder) in which the pre-conversion position information(input_knee_point) is smaller.

Further, a decoded image may be knee-converted by using an approximatefunction of knee conversion. The approximate function of knee conversionis a straight line which connects 0-th to N-th (where N is equal to orgreater than 0 and equal to or smaller than num_knee_point_minus1) kneepoints to each other in an order in which the pre-conversion positioninformation is less. Since the order i of knee points is set in an orderin which a priority for representing a desired function of kneeconversion is higher, an approximate function of knee conversion is moreapproximate to a desired function of knee conversion as N is greater.

(First Configuration Example of One Embodiment of Decoding System)

FIG. 49 is a block diagram illustrating a first configuration example ofan embodiment of a decoding system to which the present disclosure isapplied and which decodes a coded stream transmitted from the firstconfiguration of the third embodiment of the coding device to which thepresent disclosure is applied.

Among constituent elements illustrated in FIG. 49, the same constituentelements as the constituent elements of FIGS. 12 and 22 are given thesame reference numerals. Repeated description will be omitted asappropriate.

A decoding system 110 of FIG. 49 includes a decoding device 111 and adisplay device 112. The decoding device 111 includes a reception unit51, an extraction unit 91, a decoding unit 92, a selection unit 121, anda transmission unit 122.

The selection unit 121 of the decoding device 111 acquiresknee_function_info SEI among parameter sets extracted by the extractionunit 91. The selection unit 121 selects DR conversion information of thenumber (for example, 3) of knee points included in a single packet ofAVI InfoFrame of HDMI in an order in which the order i is lower fromamong DR conversion information pieces of a plurality of knee pointsincluded in the knee_function_info SEI. The selection unit 121 suppliesthe selected DR conversion information of the knee point to thetransmission unit 122.

The transmission unit 122 disposes the DR conversion informationselected by the selection unit 121 in a single packet of AVI InfoFrameof HDMI, and transmits a result thereof to the display device 112 withHDMI along with a decoded image generated by the decoding unit 92.

The display device 112 includes a reception unit 131, a conversion unit93, a display control unit 94, and a display unit 95.

The reception unit 131 of the display device 112 receives AVI InfoFrameand the decoded image which are transmitted from the transmission unit122 with HDMI. The reception unit 131 supplies the DR conversioninformation disposed in AVI InfoFrame and the decoded image to theconversion unit 93.

[Description of First Selection Method of Knee Point]

FIG. 50 is a diagram illustrating an example of a knee point and afunction of knee conversion defined by the knee_function_info SEI whichis received by the decoding system 110 of FIG. 49.

In addition, in the example of FIG. 50, a knee point number(number_knee_point_minus1) set in the knee_function_info SEI is 8.

As illustrated in FIG. 50A, among eight knee points set in theknee_function_info SEI, pre-conversion position information(Input_knee_point[0]) of the 0-th knee point is 200, and post-conversionposition information (output_knee_point[0]) thereof is 433. In addition,pre-conversion position information (input_knee_point[1]) of the firstknee point is 600, and post-conversion position information(output_knee_point[1]) thereof is 774, and pre-conversion positioninformation (input_knee_point[2]) of the second knee point is 100, andpost-conversion position information (output_knee_point[2]) thereof is290.

Further, pre-conversion position information (input_knee_point[3]) ofthe third knee point is 400, and post-conversion position information(output_knee_point[3]) thereof is 628, and pre-conversion positioninformation (input_knee_point[4]) of the fourth knee point is 800, andpost-conversion position information (output_knee_point[4]) thereof is894.

Furthermore, pre-conversion position information (input_knee_point[5])of the fifth knee point is 300, and post-conversion position information(output_knee_point[5]) thereof is 540, and pre-conversion positioninformation (input_knee_point[6]) of the sixth knee point is 500, andpost-conversion position information (output_knee_point[6])) thereof is705.

In addition, pre-conversion position information (input_knee_point[7])of the seventh knee point is 700, and post-conversion positioninformation (output_knee_point[7]) thereof is 836, and pre-conversionposition information (input_knee_point[8]) of the eighth knee point is900, and post-conversion position information (output_knee_point[8])thereof is 949.

In this case, the respective knee points are connected to each other inan order in which the pre-conversion position information is less, andthus a function of knee conversion is as illustrated in FIG. 50B. Inother words, a straight line which connects the knee points to eachother in an order of the second, 0-th, fifth, third, sixth, first,seventh, fourth and eighth knee points, serves as a function of kneeconversion. In addition, the transverse axis of FIG. 50B expressesluminance of a coding target image, and the longitudinal axis expressesluminance of a converted image. This is also the same for FIGS. 51, 52,and 57 to 59 described later.

In a case where the selection unit 121 selects DR conversion informationpieces of three knee points from the DR conversion information pieces ofthe knee points defined by the knee_function_info SEI of FIG. 50A, anapproximate function of knee conversion having the selected knee pointsis as illustrated in FIG. 51.

In other words, in this case, the selection unit 121 selects DRconversion information pieces of the 0-th to second knee points fromamong the DR conversion information pieces of the 0-th to eighth kneepoints defined by the knee_function_info SEI. Therefore, a kneeconversion function having the selected knee points is a straight linewhich connects the 0-th to second knee points to each other in an orderin which the pre-conversion position information is less, that is, in anorder of the second, 0-th and first knee points.

Meanwhile, in a case where the selection unit 121 selects DR conversioninformation pieces of five knee points from among the DR conversioninformation pieces of the knee points defined by the knee_function_infoSEI of FIG. 50A, an approximate function of knee conversion having theselected knee points is as illustrated in FIG. 52.

In other words, in this case, the selection unit 121 selects DRconversion information pieces of the 0-th to fourth knee points fromamong the DR conversion information pieces of the 0-th to eighth kneepoints defined by the knee_function_info SEI. Therefore, a kneeconversion function having the selected knee points is a straight linewhich connects the 0-th to fourth knee points to each other in an orderin which the pre-conversion position information is less, that is, in anorder of the second, 0-th, third, first and fourth knee points.

The order i of the knee points is set in an order of a priority forrepresenting the function of FIG. 50B which is a desired function ofknee conversion is higher, and DR conversion information pieces of apredetermined number of knee points are selected from the lower order i.Therefore, as illustrated in FIGS. 51 and 52, an approximate function ofknee conversion is more approximate to the function of FIG. 50B than ina case where other knee points of the same number are selected.

In addition, a larger number of knee points lead to a smoother and morecomplex function. Therefore, an approximate function of knee conversionof FIG. 52 in which the number of knee points is five is moreapproximate to the function of knee conversion of FIG. 50B than anapproximate function of knee conversion of FIG. 51 in which the numberof knee points is three.

[Description of Process in Decoding System]

FIG. 53 is a flowchart illustrating a decoding process performed by thedecoding device 111 of the decoding system 110 of FIG. 419.

In step S111 of FIG. 53, the reception unit 51 of the decoding device111 receives a coded stream transmitted from the coding device 70 ofFIG. 16, and supplies the coded stream to the extraction unit 91.

In step S112, the extraction unit 91 extracts parameter sets and codeddata from the coded stream which is supplied from the reception unit 51.The extraction unit 91 supplies the parameter sets and the coded data tothe decoding unit 92. In addition, the extraction unit 91 suppliesknee_function_info SEI among the parameter sets to the selection unit121.

In step S113, the decoding unit 92 decodes the coded data supplied fromthe extraction unit 91 in the HEVC method. At this time, the decodingunit 92 also refers to the parameter sets supplied from the extractionunit 91 as necessary. The decoding unit 92 supplies the decoded image tothe transmission unit 122.

In step S114, the selection unit 121 selects DR conversion informationof the number of knee points included in a single packet of AVIInfoFrame of HDMI in an order in which the order i is lower from amongDR conversion information pieces of a plurality of knee points includedin the knee_function_info SEI from the extraction unit 91. The selectionunit 121 supplies the selected DR conversion information of the kneepoint to the transmission unit 122.

In step S115, the transmission unit 122 disposes the DR conversioninformation selected by the selection unit 121 in a single packet of AVIInfoFrame of HDMI, and transmits a result thereof to the display device112 with HDMI along with a decoded image generated by the decoding unit92. In addition, the process is finished.

FIG. 54 is a flowchart illustrating a display process performed by thedisplay device 112 of the decoding system 110.

In step S131 of FIG. 54, the reception unit 131 of the display device112 receives the DR conversion information disposed in AVI InfoFrame andthe decoded image which are transmitted from the transmission unit 122with HDMI. The reception unit 131 supplies the DR conversion informationand the decoded image to the conversion unit 93.

Processes in steps S132 to S134 are the same as the processes in stepsS95 and S97 of FIG. 23, and thus description thereof will not berepeated.

As mentioned above, in the first configuration of the third embodimentto which the present disclosure is applied, the DR conversioninformation of the knee point in which the order is set in an order inwhich a priority for representing a desired knee conversion is higher isset in the knee_function_info SEI and is transmitted. Therefore, thedecoding device 111 selects DR conversion information of the number ofknee points included in a single packet of AVI InfoFrame in an order inwhich the order i is lower, and thus can dispose DR conversioninformation of the knee point indicating an approximate function of kneeconversion which is more approximate to a desired function of kneeconversion in a single packet of AVI InfoFrame.

(Example of Syntax of Knee_Function_Info SEI)

A second configuration of the third embodiment of the coding device towhich the present disclosure is applied is the same as the configurationof the coding device 70 of FIG. 16 except for the knee_function_info SEIset by the setting unit 71 and semantics. Therefore, hereinafter, onlythe knee_function_info SEI and semantics will be described.

FIG. 55 is a diagram illustrating an example of syntax ofknee_function_info SEI set by the setting unit 71 in the secondconfiguration of the third embodiment of the coding device to which thepresent disclosure is applied.

The knee_function_info SEI of FIG. 55 is the same as theknee_function_info SEI of FIG. 40 except that an approximate knee pointindex (approximate_knee_point_index) (priority information) indicatingthe order i is set in an order in which a priority for representing adesired function of knee conversion is higher.

In the knee_function_info SEI of FIG. 55, the order i of knee points isan order in which the pre-conversion position information is less in thesame manner as in the case of FIG. 40, but the approximate knee pointindex (approximate_knee_point_index) is newly set. A value of theapproximate knee point index (approximate_knee_point_index) is equal toor less than the knee point number (number_knee_point_minus1).

(Second Example of Semantics)

FIG. 56 is a diagram illustrating that semantics of FIG. 55 is differentfrom that of the second embodiment.

As illustrated in FIG. 56, in the semantics of FIG. 55, a decoded imagemay be knee-converted by using an approximate function of kneeconversion. This approximate function of knee conversion is a straightline which connects knee points in which the order i is 0-th to N-th(where N is equal to or greater than 0 and equal to or smaller thannum_knee_point_minus1) approximate knee point index(approximate_knee_point_index[0] to approximate_knee_point_index[N]) inan order in which the order i is lower. An order j of the approximateknee point indexes is an order in which a priority for representing adesired function of knee conversion is higher, and thus an approximatefunction of knee conversion is more approximate to a desired function ofknee conversion as N is greater.

(Configuration Example of One Embodiment of Coding System)

A second configuration of an embodiment of the decoding system to whichthe present disclosure is applied is the same as the configuration ofthe decoding system 110 of FIG. 49 except that selection by theselection unit 121 is performed on the basis of not the order i of kneepoints but the order j of the approximate knee point indexes. Therefore,hereinafter, only selection by the selection unit 121 will be described.

[Description of Second Selection Method of Knee Point]

FIGS. 57A and 57B are diagrams illustrating an example of a knee pointand a function of knee conversion defined by the knee_function_info SEIof FIG. 55.

In addition, in the example of FIGS. 57A and 57B, a knee point number(number_knee_point_minus1) set in the knee_function_info SEI is 8 in thesame manner as in FIGS. 50A and 50B, and knee points are also the sameas in FIGS. 50A and 50B. However, in the knee_function_info SEI of FIG.55, the order i of knee points is an order in which the pre-conversionposition information is less, and is thus different from that of FIGS.50A and 50B.

As illustrated in FIG. 57A, among eight knee points set in theknee_function_info SEI, pre-conversion position information(input_knee_point[0]) of the 0-th knee point is 100, and post-conversionposition information (output_knee_point[0]) thereof is 290. In addition,pre-conversion position information (input_knee_point[1]) of the firstknee point is 200, and post-conversion position information(output_knee_point[1]) thereof is 433, and pre-conversion positioninformation (input_knee_point[2]) of the second knee point is 300, andpost-conversion position information (output_knee_point[2]) thereof is540.

Further, pre-conversion position information (input_knee_point[3]) ofthe third knee point is 400, and post-conversion position information(output_knee_point[3]) thereof is 628, and pre-conversion positioninformation (input_knee_point[4]) of the fourth knee point is 500, andpost-conversion position information (output_knee_point[4]) thereof is705.

Furthermore, pre-conversion position information (input_knee_point[5])of the fifth knee point is 600, and post-conversion position information(output_knee_point[5]) thereof is 774, and pre-conversion positioninformation (input_knee_point[6]) of the sixth knee point is 700, andpost-conversion position information (output_knee_point[6]) thereof is836.

In addition, pre-conversion position information (input_knee_point[7])of the seventh knee point is 800, and post-conversion positioninformation (output_knee_point[7]) thereof is 894, and pre-conversionposition information (input_knee_point[8]) of the eighth knee point is900, and post-conversion position information (output_knee_point[8])thereof is 949.

In this case, the respective knee points are connected to each other inan order in which the order i is lower, and thus a function of kneeconversion is as illustrated in FIG. 57B.

In addition, as illustrated in FIG. 57A, the approximate knee pointindexes (approximate_knee_point_index) in which the order j is 0 to 8are 1, 5, 0, 3, 7, 2, 4, 6, and 8 in order.

In a case where the selection unit 121 selects DR conversion informationpieces of three knee points from among the DR conversion informationpieces of the knee points defined by the knee_function_info SEI of FIG.57A, a function of knee conversion having the selected knee points is asillustrated in FIG. 58.

In other words, in this case, the selection unit 121 selects DRconversion information pieces of the knee points in which the order i isthe 0-th to second approximate knee point indexes(approximate_knee_point_index) from among the DR conversion informationpieces of the 0-th to eighth knee points defined by theknee_function_info SEI. In other words, the selection unit 121 selectsthe DR conversion information pieces of the first, fifth and 0-th kneepoints. Therefore, a knee conversion function having the selected kneepoints is a straight line which connects the first, fifth and 0-th kneepoints to each other in an order in which the order is lower, that is,in an order of the 0-th, first and fifth knee points.

Meanwhile, in a case where the selection unit 121 selects DR conversioninformation pieces of five knee points from among the DR conversioninformation pieces of the knee points defined by the knee_function_infoSEI of FIG. 57A, a function of knee conversion having the selected kneepoints is as illustrated in FIG. 59.

In other words, in this case, the selection unit 121 selects DRconversion information pieces of the knee points in which the order i isthe 0-th to fourth approximate knee point indexes(approximate_knee_point_index) from among the DR conversion informationpieces of the 0-th to eighth knee points defined by theknee_function_info SEI. In other words, the selection unit 121 selectsthe DR conversion information pieces of the first, fifth, 0-th, thirdand seventh knee points. Therefore, a knee conversion function havingthe selected knee points is a straight line which connects the first,fifth, 0-th, third and seventh knee points to each other in an order inwhich the order i is lower, that is, in an order of the 0-th, first,third, fifth and seventh knee points.

The order j of the approximate knee point indexes is set in an order ofpriorities for representing the function of FIG. 57B which is a desiredfunction of knee conversion is higher, and DR conversion informationpieces of knee points with a predetermined number of approximate kneepoint indexes in the order i are selected from the lower order j.Therefore, as illustrated in FIGS. 58 and 59, an approximate function ofknee conversion is more approximate to the function of FIG. 57B than ina case where other knee points of the same number are selected.

In addition, a larger number of knee points lead to a smoother and morecomplex function. Therefore, an approximate function of knee conversionof FIG. 59 in which the number of knee points is five is moreapproximate to the function of knee conversion of FIG. 57B than anapproximate function of knee conversion of FIG. 58 in which the numberof knee points is three.

In addition, as illustrated in FIG. 60, the approximate knee point index(approximate_knee_point_index) may be set inapproximate_knee_function_info SEI different from knee_function_infoSEI.

In this case, an approximate knee conversion ID(approximate_knee_function_id) and an approximate knee conversion cancelflag (approximate_knee_function_cancel_flag) are set in theapproximate_knee_function_info SEI.

The approximate knee conversion ID is an ID unique to the purpose ofknee conversion using an approximate function. In addition, theapproximate knee conversion cancel flag is a flag illustrating whetheror not persistence of previous approximate_knee_function_info SEI iscanceled. The approximate knee conversion cancel flag is set to 1 whenindicating that persistence of previous approximate_knee_function_infoSEI is canceled, and is set to 0 when the persistence is not canceled.

In a case where the approximate knee conversion cancel flag is 0, areference knee conversion ID (ref_knee_function_id) is set in theapproximate_knee_function_info SEI. The reference knee conversion ID isa knee conversion ID of knee_function_info SEI including DR informationof a knee point indicating a function of knee conversion which isapproximated by using an approximate knee point index of theapproximate_knee_function_info SEI.

In addition, an approximate knee point index number(num_approximate_knee_point_indices_minus1) which is a value obtained bysubtracting 1 from the number of approximate knee point indexes, and anapproximately knee point index (approximate_knee_point_index) are set.

As mentioned above, also in a case where the approximate knee pointindex (approximate_knee_point_index) is set in theapproximate_knee_function_info SEI, semantics is the same as thesemantics described in FIG. 56.

In addition, in the above description, only the knee_function_info SEIincluding DR information of a knee point indicating a function of kneeconversion is set, but knee_function_info SEI including DR informationof a knee point indicating an approximate function of knee conversionmay be set. In this case, for example, DR information of a knee pointindicating a function of knee conversion is set to knee_function_infoSEI in which a knee conversion ID is 0, and DR information of a kneepoint indicating an approximate function of knee conversion is set toknee_function_info SEI in which a knee conversion ID is 1. Further, in acase where DR information is transmitted with HDMI, the decoding devicedisposes the DR information included in the knee_function_info SEI inwhich the knee conversion ID is 1, in a single packet of AVI InfoFrame,and transmits the DR information.

In addition, a unique ID is set in predetermined brightness as theunconverted display luminance information (input_disp_luminance) and theconverted luminance range information (output_d_range), and thus it ispossible to reduce a DR information amount. In this case, for example, 0may be assigned to 400 candela per square meter, and 1 may be assignedto 800 candela per square meter, as an ID. A correspondence relationshipbetween an ID and brightness assigned with the ID is set in common to acoding side and a display side, and thus the display side can recognizethe brightness from the ID.

In the third embodiment, a knee point is selected in an order in whichpriorities for representing a desired function of knee conversion arehigher, but a knee point may be selected in other orders.

In addition, in the third embodiment, the number of selected knee pointsis the number which can be included in a single packet of AVI InfoFrame,but is not limited thereto. For example, in a case where the decodingdevice 111 has a function of the display device 112, the number ofselected knee points may be the number of knee points corresponding toknee conversion which can be processed by the conversion unit 93, or thelike.

[Fourth Embodiment]

[Basis of Fourth Embodiment]

As illustrated in FIG. 61, in a cathode ray tube (CRT) used in a CRTdisplay, an input electrical signal and display luminance have noproportional relationship, and it is necessary to input a higherelectrical signal in order to display high luminance. Therefore, if anelectrical signal which is proportional to luminance of an image isinput to the CRT display as illustrated in FIG. 62, display luminance islower than original luminance of the image as illustrated in FIG. 63.Therefore, in order to display an image with the original luminance ofthe image, as illustrated in FIG. 64, it is necessary to convertluminance of an image into an electrical signal by using a functionhaving a characteristic reverse to that of the function of FIG. 61.

In addition, in FIGS. 61 and 63, the transverse axis expresses a valueobtained by normalizing an input electrical signal when a value of theinput electrical signal for displaying with the maximum luminance in theCRT display is set to 1, and the longitudinal axis expresses a valueobtained by normalizing display luminance when the maximum value of thedisplay luminance of the CRT display is set to 1. In FIGS. 62 and 64,the transverse axis expresses a value obtained by normalizing luminanceof a display target image when the maximum value of the luminance of adisplay target image is set to 1, and the longitudinal axis expresses avalue obtained by normalizing an electrical signal when a value of theelectrical signal corresponding to the maximum value of the luminance ofa display target image is set to 1.

A function for converting an input electrical signal into displayluminance as illustrated in FIG. 61 is referred to as electro-opticaltransfer function (EOTF), and a function for converting luminance of animage into an electrical signal as illustrated in FIG. 64 is referred toas an optical-electro transfer function (OETF).

Other displays such as a light emitting diode (LED) panel havecharacteristics different from the characteristics of the CRT display.However, in order not to change generation procedures of an inputelectrical signal depending on displays, processes using the EOTF andthe OETF are also performed in the same manner as in the CRT display ina case of performing display with other displays.

FIG. 65 is a diagram illustrating an example of a flow of a processuntil an image is displayed from capturing of the image.

In addition, in the example of FIG. 65, an electrical signal is a codevalue of 10 bits (0 to 1023), and the OETF and the EOFT are defined inBT.709.

As illustrated in FIG. 65, when an image is captured by a camera or thelike, a photoelectric conversion process of converting luminance (light)into an electrical signal (code value) by using the OETF is performed onthe captured image. Then, the electrical signal is coded, and the codedelectrical signal is decoded. In addition, an electro-optical conversionprocess of converting an electrical signal into luminance by using theEOTF is performed on the decoded electrical signal.

Meanwhile, the human visual sense has a characteristic of beingsensitive to a luminance difference at low luminance and beinginsensitive to a luminance difference at high luminance. Therefore, asillustrated in FIG. 65, the OETF of BT.709 is a function in which morecode values are assigned to a low luminance part than a high luminancepart. As a result, subjectively sufficient image quality is realized.

In a case where the maximum luminance of an image is about 100 candelaper square meter, satisfactory code values can be assigned to a lowluminance part by using the OETF of BT.709. However, the maximumluminance of displays has recently tended to increase, and is expectedto be accelerated in the future. If the maximum luminance of an imageincreases in accordance therewith, code values to be assigned to a lowluminance part are insufficient in the OETF of BT.709, and thussatisfactory image quality is unable to be obtained.

Therefore, it is considered that a new OETF for use in an HDR image inwhich a ratio of code values assigned to a low luminance part isincreased is generated, and thus satisfactory image quality is obtainedin an HDR image. However, in this case, in order to perform aphotoelectric conversion process and an electro-optical conversionprocess, it is necessary to prepare for both an OETF and an EOTF for anHDR image and an OETF and an EOTF for an SDR image.

On the other hand, in a case where electro-optical conversion isperformed on an SDR image by using an OETF for an HDR image, grayscaleexpression of luminance is roughened.

For example, as illustrated in FIG. 66, in a case where photoelectricconversion is performed on an SDR image by using an OETF of BT.709 foran SDR image having the maximum luminance of 100 candela per squaremeter, luminance of the SDR image is expressed in codes of 1024including 0 to 1023. In contrast, as illustrated in FIG. 67, in a casewhere photoelectric conversion is performed on an SDR image by using anOETF for an HDR image having the maximum luminance of 400 candela persquare meter, luminance of the SDR image is expressed, for example, in502 code values including 0 to 501.

Therefore, an OETF and an EOTF are preferably variable in order toassign sufficient code values to a low luminance part in both an HDRimage having high maximum luminance and an SDR image having low maximumluminance. Therefore, in the fourth embodiment, knee conversion isperformed before the OETF of BT.709 and after the EOTF of BT.709, andthus sufficient code values can be assigned to a low luminance part.

[Overview of Photoelectric Conversion Process in Fourth Embodiment]

FIG. 68 is a diagram illustrating an overview of a photoelectricconversion process in the fourth embodiment.

As illustrated in the left part of FIG. 68, in the fourth embodiment,first, predetermined knee conversion is performed on luminance (inputluminance) of a captured image. In an example of FIG. 68, through theknee conversion, 10% of a low luminance part of the input luminance isconverted into 90% of the low luminance part of input luminance′, and90% of a high luminance part of the input luminance is converted into10% of the high luminance part of the input luminance′. Accordingly,there is a generation of the input luminance′ in which more values areassigned to the low luminance part than the high luminance part.

Next, as illustrated in the central part of FIG. 68, a photoelectricconversion process using the OETF of BT.709 is performed on the inputluminance′ so as to generate a code value of a predetermined number ofbits (10 bits in the example of FIG. 68). As described above, since, inthe input luminance′, more values are assigned to the low luminance partthan the high luminance part, as illustrated in the right part of FIG.68, more values are assigned in code values converted from the inputluminance′ due to the low luminance part of the input luminance than inthe OETF of BT.709. In the example of FIG. 68, 10% of the low luminancepart of the input luminance is assigned to 94% of code values.

As mentioned above, in the fourth embodiment, an extent of assigningcode values to a low luminance part (dark part) and an extent ofassigning the code values to a high luminance part (bright part) areadjusted by using a function of knee conversion as a parameter.

In addition, information on a knee point of knee conversion performed onthe input luminance is set in the knee_function_info SEI of FIG. 40 andis transmitted to a decoding side.

[Overview of Electro-Optical Conversion Process in Fourth Embodiment]

FIG. 69 is a diagram illustrating an overview of an electro-opticalconversion process in the fourth embodiment.

As illustrated in the left part of FIG. 69, in the fourth embodiment,first, an electro-optical conversion process using the EOTF of BT.709 isperformed on code values of a decoded image so as to generate luminance(output luminance). Next, as illustrated in the central part of FIG. 69,predetermined knee conversion is performed on the output luminance. Inan example of FIG. 68, through the knee conversion, 90% of a lowluminance part of the output luminance is converted into 10% of a lowluminance part of output luminance′, and 10% of a high luminance part ofthe output luminance is converted into 90% of a high luminance part ofthe output luminance′.

Accordingly, as illustrated in the right part of FIG. 69, code values inwhich more values are assigned due to the low luminance part of theinput luminance than in the EOTF of BT.709 can be converted into thesame output luminance′ as input luminance corresponding to the codevalues.

As mentioned above, in the fourth embodiment, code values in which anextent of assignment to a low luminance part (dark part) and an extentof assignment to a high luminance part (bright part) are adjusted areconverted into luminance by using a function of knee conversion as aparameter.

In addition, information on a knee point of knee conversion performed onthe output luminance is determined on the basis of information set inknee_function_info SEI or the like transmitted from a coding side.

(Configuration Example of Fourth Embodiment of Coding Device)

FIG. 70 is a block diagram illustrating a configuration example of thefourth embodiment of a coding device to which the present disclosure isapplied.

Among constituent elements illustrated in FIG. 70, the same constituentelements as the constituent elements of FIG. 6 or 16 are given the samereference numerals. Repeated description will be omitted as appropriate.

A configuration of a coding device 150 of FIG. 70 is different from theconfiguration of FIG. 16 in that a quantization unit 151 is providedinstead of the conversion unit 73. The coding device 150 performs aphotoelectric conversion process on a captured image which is input froman external device so as to perform coding.

Specifically, the quantization unit 151 of the coding device 150knee-converts luminance of the captured image which is input from theexternal device. Information on a knee point of the knee conversion isset in knee_function_info SEI by the setting unit 71. The quantizationunit 151 performs a photoelectric conversion process using the OETF ofBT.709 on the knee-converted luminance so as to generate a code value.The quantization unit 151 supplies the generated code value to thecoding unit 72 as a coding target image.

<Description of Process in Coding Device>

FIG. 71 is a flowchart illustrating a stream generation processperformed by the coding device 150 of FIG. 70.

In step S150 of FIG. 71, the quantization unit 151 of the coding device150 knee-converts luminance of a captured image which is input from anexternal, device. In step S152, the quantization unit 151 performs aphotoelectric conversion process using the EOTF of BT.709 on theknee-converted luminance so as to generate a code value. Thequantization unit 151 supplies the generated code value to the codingunit 72 as a coding target image.

Processes in steps S152 to S154 are the same as the processes in stepsS73 to S75 of FIG. 21, and thus description thereof will be omitted.

In step S155, the setting unit 71 sets knee_function_info SEI includinginformation on a knee point of the knee conversion performed due to theprocess in step S150. The setting unit 71 supplies parameter sets suchas the set SPS, PPS, VUI and knee_function_info SEI to the coding unit72.

In step S156, the coding unit 72 codes the coding target image which issupplied from the conversion unit 73 in the HEVC method. Processes insteps S157 and S158 are the same as the processes in steps S78 and S79of FIG. 21, and thus description thereof will be omitted.

As mentioned above, the coding device 150 performs the knee conversionbefore the OETF of BT.709, and thus can perform a photoelectricconversion process suitable for both an SDR image and an HDR image byusing the OETF of BT.709.

(Configuration Example of Fourth Embodiment of Decoding Device)

FIG. 72 is a block diagram illustrating a configuration example of thefourth embodiment of a decoding device to which the present disclosureis applied and which decodes a coded stream transmitted from the codingdevice 150 of FIG. 70.

Among constituent elements illustrated in FIG. 72, the same constituentelements as the constituent elements of FIG. 12 or 22 are given the samereference numerals. Repeated description will be omitted as appropriate.

A configuration of a decoding device 170 of FIG. 72 is different fromthe configuration of the decoding device 90 of FIG. 22 in that aconversion unit 171 is provided instead of the conversion unit 93. Thedecoding device 170 decodes a coded stream, and performs anelectro-optical conversion process on a decoded image which is obtainedas a result thereof.

Specifically, the conversion unit 171 of the decoding device 170performs an electro-optical conversion process using the EOTF of BT.709on a code value as a decoded image supplied from the decoding unit 92,so as to generate luminance. The conversion unit 171 performs kneeconversion on the luminance on the basis of knee_function_info SEI fromthe extraction unit 91. The conversion unit 171 supplies luminance whichis obtained as a result of the knee conversion to the display controlunit 94 as a display image.

<Description of Process in Decoding Device>

FIG. 73 is a flowchart illustrating an image generation processperformed by the decoding device 170 of FIG. 72.

Processes in steps S171 to S173 of FIG. 73 are the same as the processesin steps S91 to S93 of FIG. 23, and thus description thereof will beomitted.

In step S174, the conversion unit 171 of the decoding device 170performs an electro-optical conversion process using the EOTF of BT.709on a code value as a decoded image supplied from the decoding unit 92,so as to generate luminance.

In step S175, the conversion unit 171 performs knee conversion on thegenerated luminance on the basis of knee_function_info SEI from theextraction unit 91. The conversion unit 171 supplies luminance which isobtained as a result of the knee conversion to the display control unit94 as a display image.

In step S176, the display control, unit 94 displays the display imagesupplied from the conversion unit 93 on the display unit 95, andfinishes the process.

As mentioned above, the decoding device 170 performs the knee conversionafter the EOTF of BT.709, and thus can perform an electro-opticalconversion process suitable for both an SDR image and an HDR image byusing the EOTF of BT.709.

In addition, the maximum luminance of a coding target image may beincluded in a coded stream along with coded data and may be transmittedto the decoding device 170 from the coding device 150, and may bedetermined in advance as a value common to the coding device 150 and thedecoding device 170. Further, knee_function_info SEI may be set for eachitem of the maximum luminance of a coding target image.

In addition, in the fourth embodiment, the knee_function_info SEI of thefirst to third embodiments may be set. In this case, the decoding sideperforms knee conversion by using DR conversion information, and thus itis possible to perform conversion into an image which is suitable forvarious luminance displays.

In addition, the decoding device 170 in the fourth embodiment may bedivided into a decoding device and a display device in the same manneras in the third embodiment.

Further, in the fourth embodiment, an extent of assigning code values toa low luminance part and an extent of assigning the code values to ahigh luminance part are adjusted by using a function of knee conversionas a parameter, but may be adjusted by using functions other than thefunction of knee conversion as a parameter.

Furthermore, the present disclosure may be applied to the AVC method.

[Fifth Embodiment]

[Description of Computer to which Present Disclosure is Applied]

The above-described series of processes may be performed by hardware orsoftware. When the above-described series of processes is performed bythe software, programs constituting the software are installed in acomputer. Here, the computer includes a computer incorporated intodedicated hardware, or a general purpose personal computer or the likewhich can execute various kinds of functions by installing various kindsof programs.

FIG. 74 is a block diagram illustrating a configuration example ofhardware of a computer which performs the above-described series ofprocesses according to a program.

In the computer, a central processing unit (CPU) 201, a read only memory(ROM) 202, and a random access memory (RAM) 203 are connected to eachother via a bus 204.

The bus 204 is also connected to an input and output interface 205. Theinput and output interface 205 is connected to an input unit 206, anoutput unit 207, a storage unit 208, a communication unit 209, and adrive 210.

The input unit 206 includes a keyboard, a mouse, a microphone, and thelike. The output unit 207 includes a display, a speaker, and the like.The storage unit 208 includes a hard disk, a nonvolatile memory, or thelike. The communication unit 209 includes a network interface or thelike. The drive 210 drives a removable medium 211 such as a magneticdisk, an optical disc, a magneto-optical disc, or the like.

In the computer configured in this way, the CPU 201 performs theabove-described series of processes, for example, by loading the programstored in the storage unit 208 to the RAM 203 via the input and outputinterface 205 and the bus 204 and executing the program.

The program executed by the computer (the CPU 201) may be recorded onthe removable medium 211, for example, as a package medium, and may beprovided. In addition, the program may be provided via a wired orwireless transmission medium such as a local area network, the Internet,or a digital satellite broadcast.

In the computer, the program may be installed in the storage unit 208via the input and output interface 205 by installing the removablemedium 211 in the drive 210. In addition, the program may be received bythe communication unit 209 via a wired or wireless transmission mediumand may be installed in the storage unit 208. Further, the program maybe installed in the ROM 202 or the storage unit 208 in advance.

In addition, the program executed by the computer may be a program whichperforms processes in a time series according to the order described inthe present specification, and may be a program which performs processesin parallel or at a necessary timing such as when accessed.

[Sixth Embodiment]

[Application to Multi-View Image Coding and Multi-View Image Decoding]

The above-described series of processes may be applied to multi-viewimage coding and multi-view image decoding. FIG. 75 is a diagramillustrating an example of a multi-view image coding method.

As illustrated in FIG. 75, multi-view images include images at aplurality of views. The plurality of views of the multi-view imagesincludes a base view in which coding/decoding is performed by using onlyan image at its own view, and a non-base view in which coding/decodingis performed by using images at other views. The non-base view may use abase view image and may use other non-base view images.

In a case of coding/decoding multi-view images as in FIG. 75, each viewimage is coded/decoded, and the above-described method of the firstembodiment may be applied to coding/decoding of each view. In this way,a decoded image can be converted into a desired image with a differentdynamic range.

In addition, in coding/coding of each view, the flag or the parameterused in the method of the first embodiment may be shared. Morespecifically, for example, the syntax element or the like of theknee_function_info SEI may be shared in coding/decoding of each view. Ofcourse, necessary information other than these elements may be shared incoding/decoding of each view.

In this way, it is possible to minimize transmission of redundantinformation and thus to reduce transmitted information amount (bit rate)(that is, it is possible to minimize a reduction in coding efficiency).

[Multi-View Image Coding Device]

FIG. 76 is a diagram illustrating a multi-view image coding device whichperforms the above-described multi-view image coding. As illustrated inFIG. 76, the multi-view image coding device 600 includes a coding unit601, a coding unit 602, and a multiplexer 603.

The coding unit 601 codes a base view image so as to generate a baseview image coded stream. The coding unit 602 codes a non-base view imageso as to generate a non-base view image coded stream. The multiplexer603 multiplexes the base view image coded stream generated in the codingunit 601 and the non-base view image coded stream generated in thecoding unit 602, so as to generate a multi-view image coded stream.

The coding device 10 (FIG. 6) is applicable to the coding unit 601 andthe coding unit 602 of the multi-view image coding device 600. In otherwords, in coding of each view, an image can be coded so that a decodedimage can be converted into a desired image with a different dynamicrange during decoding. In addition, the coding unit 601 and the codingunit 602 can perform coding (that is, a flag or a parameter can beshared) by using the mutually same flags or parameters (for example, asyntax element or the like regarding a process of images), and thus itis possible to minimize a reduction in coding efficiency.

[Multi-View Image Decoding Device]

FIG. 77 is a diagram illustrating a multi-view image decoding devicewhich performs the above-described multi-view image decoding. Asillustrated in FIG. 77, the multi-view image decoding device 610includes a demultiplexer 611, a decoding unit 612, and a decoding unit613.

The demultiplexer 611 demultiplexes the multi-view image coded streaminto which the base view image coded stream and the non-base view imagecoded stream are multiplexed, so as to extract the base view image codedstream and the non-base view image coded stream. The decoding unit 612decodes the base view image coded stream extracted by the demultiplexer611 so as to obtain a base view image. The decoding unit 613 decodes thenon-base view image coded stream extracted by the demultiplexer 611 soas to obtain a non-base view image.

The decoding device 50 (FIG. 12) is applicable to the decoding unit 612and the decoding unit 613 of the multi-view image decoding device 610.In other words, in decoding of each view, a decoded image can beconverted into a desired image with a different dynamic range. Inaddition, the decoding unit 612 and the decoding unit 613 can performcoding (that is, a flag or a parameter can be shared) by using themutually same flags or parameters (for example, a syntax element or thelike regarding a process of images), and thus it is possible to minimizea reduction in coding efficiency.

[Seventh Embodiment]

[Application to Layer Image Coding and Layer Image Decoding]

The above-described series of processes may be applied to layer imagecoding and layer image decoding. FIG. 78 illustrates an example of alayer image coding method.

The layer image coding (scalable coding) is to generate a plurality oflayers of an image and to code each layer so that image data has ascalable function with respect to a predetermined parameter. The layerimage decoding (scalable decoding) is decoding corresponding to thelayer image coding.

As illustrated in FIG. 78, in layering of an image, a single image isdivided into a plurality of images (layers) with a predeterminedparameter having a scalable function as a reference. In other words,layered images (layer images) include images of a plurality of layers inwhich values of the predetermined parameter are different from eachother. A plurality of layers of the layer images include a base layer inwhich coding/decoding is performed by using only an image of its ownlayer and a non-base layer (also referred to as an enhancement layer) inwhich coding/decoding is performed by using images of other layers. Thenon-base layer may use a base layer image and may use other non-baselayer images.

Generally, the non-base layer is formed by its own image and data(difference data) on a difference image with images of other layers. Forexample, in a case where a single image is generated as two layersincluding a base layer and a non-base layer (also referred to as anenhancement layer), an image with quality lower than that of anoriginal, image is obtained only by using data of the base layer, andthus data of the base layer and data of the non-base layer are combinedwith each other so as to obtain the original image (that is, highquality image).

An image is layered as mentioned above, and thus various quality imagescan be easily obtained depending on circumstances. For example, imagecompression information of only a base layer is transmitted to aterminal having low processing performance, such as a mobile phone, sothat a moving image of which spatial and temporal resolution is low orimage quality is low is reproduced, and image compression information ofan enhancement layer as well as a base layer is transmitted to aterminal with high processing performance, such as a television set or apersonal computer, so that a moving image of which spatial and temporalresolution is high or image quality is high is reproduced. In this way,image compression information can be transmitted from a server dependingon a terminal or network performance without performing a transcodeprocess.

A layer image as in the example of FIG. 78 is coded/decoded, an image ofeach layer Is coded/decoded, the above-described method of the firstembodiment may be applied to coding/decoding of each layer. In this way,a decoded image can be converted into a desired image with a differentdynamic range.

In addition, in coding/coding of each layer, the flag or the parameterused in the method of the first embodiment may be shared. Morespecifically, for example, the syntax element or the like of theknee_function_info SEI may be shared in coding/decoding of each layer.Of course, necessary information other than these elements may be sharedin coding/decoding of each layer.

In this way, it is possible to minimize transmission of redundantinformation and thus to reduce transmitted information amount (bit rate)(that is, it is possible to minimize a reduction in coding efficiency).

[Scalable Parameters]

In such layer image coding and layer image decoding (scalable coding andscalable decoding), a parameter having a scalable function is arbitrary.For example, a spatial resolution as illustrated in FIG. 79 may be aparameter (spatial scalability). In a case of the spatial scalability, aresolution of an image is different for each layer. In other words, inthis case, as illustrated in FIG. 79, each picture is generated as twolayers including a base layer of which a spatial resolution is lowerthan that of an original image, and an enhancement layer which allows anoriginal spatial resolution to be obtained through combination with thebase layer. Of course, the number of layers is an example, and anynumber of layers may be generated.

In addition, as a parameter which gives such scalability, for example, atemporal resolution may be employed (temporal scalability) asillustrated in FIG. 80. In a case of the temporal scalability, a framerate is different for each layer. In other words, in this case, asillustrated in FIG. 80, each picture is generated as two layersincluding a base layer of which a frame rate is lower than that of anoriginal moving image, and an enhancement layer which allows an originalframe rate to be obtained through combination with the base layer. Ofcourse, the number of layers is an example, and any number of layers maybe generated.

Further, as a parameter which gives such scalability, for example, asignal to noise ratio (SNR) may be employed (SNR scalability). In a caseof the SNR scalability, an SNR is different for each layer. In otherwords, in this case, as illustrated in FIG. 81, each picture isgenerated as two layers including a base layer of which an SNR is lowerthan that of an original image, and an enhancement layer which allows anoriginal SNR to be obtained through combination with the base layer. Ofcourse, the number of layers is an example, and any number of layers maybe generated.

Parameters which give scalability may use parameters other than theabove-described examples. For example, as a parameter which givesscalability, a bit depth may be used (bit-depth scalability). In a caseof the bit-depth scalability, a bit depth is different for each layer.In this case, for example, a base layer is formed by an 8-bit image, andan enhancement layer is added thereto so that a 10-bit image can beobtained.

In addition, as a parameter which gives scalability, a chroma format maybe used (chroma scalability). In a case of the chroma scalability, achroma format is different for each layer. In this case, for example, abase layer is formed by a component image with a 4:2:0 format, and anenhancement layer is added thereto so that a component image with a4:2:2 format can be obtained.

Further, as a parameter which gives scalability, a dynamic range ofluminance may be used (DR scalability). In a case of the DR scalability,a dynamic range of luminance is different for each layer. In this case,for example, a base layer is formed by an SDR image, and an enhancementlayer is added thereto so that an HDR image can be obtained.

In a case of applying the above-described series of processes to thedynamic range scalability, for example, information regarding kneedecompression from an SDR image to an HDR image is set in a coded streamof a base layer image as DR conversion information. In addition,information regarding knee compression of a dynamic range of luminanceof a HDR image is set in a coded stream of an enhancement layer image asDR conversion information.

In addition, a decoding device, which can decode only a coded stream ofa base layer image and includes an HDR display, converts an SDR imagewhich is a decoded image into an HDR image on the basis of the DRconversion information, and sets the HDR image as a display image. Onthe other hand, a decoding device, which can also decode a coded streamof an enhancement layer image and includes an HDR display which candisplay an HDR image with a low dynamic range, knee-compresses a dynamicrange of luminance of an HDR image which is a decoded image on the basisof the DR conversion information, and sets a result thereof as a displayimage.

Further, information on decompression of a dynamic range of luminance ofan HDR image may be set in a coded stream of an enhancement layer imageas DR conversion information. In this case, a decoding device, which canalso decode a coded stream of an enhancement layer image and includes anHDR display which can display an HDR image with a high dynamic range,knee-decompresses a dynamic range of luminance of an HDR image which isa decoded image on the basis of the DR conversion information, and setsa result thereof as a display image.

As mentioned above, the DR conversion information is set in a codedstream of a base layer image or an enhancement layer image, and thus itis possible to display an image which is more suitable for displayperformance.

[Layer Image Coding Device]

FIG. 82 is a diagram illustrating a layer image coding device whichperforms the above-described layer image coding. As illustrated in FIG.82, the layer image coding device 620 includes a coding unit 621, acoding unit 622, and a multiplexer 623.

The coding unit 621 codes a base layer image so as to generate a baselayer image coded stream. The coding unit 622 codes a non-base layerimage so as to generate a non-base layer image coded stream. Themultiplexer 623 multiplexes the base layer image coded stream generatedin the coding unit 621 and the non-base layer image coded streamgenerated in the coding unit 622, so as to generate a layer image codedstream.

The coding device 10 (FIG. 6) is applicable to the coding unit 621 andthe coding unit 622 of the layer image coding device 620. In otherwords, in coding of each layer, an image can be coded so that a decodedimage can be converted into a desired image with a different dynamicrange during decoding. In addition, the coding unit 621 and the codingunit 622 can perform control or the like of an intra-prediction filterprocess (that is, a flag or a parameter can be shared) by using themutually same flags or parameters (for example, a syntax element or thelike regarding a process of images), and thus it is possible to minimizea reduction in coding efficiency.

[Layer Image Decoding Device]

FIG. 83 is a diagram illustrating a layer image decoding device whichperforms the above-described layer image decoding. As illustrated inFIG. 83, the layer image decoding device 630 includes a demultiplexer631, a decoding unit 632, and a decoding unit 633.

The demultiplexer 631 demultiplexes the layer image coded stream intowhich the base layer image coded stream and the non-base layer imagecoded stream are multiplexed, so as to extract the base layer imagecoded stream and the non-base layer image coded stream. The decodingunit 632 decodes the base layer image coded stream extracted by thedemultiplexer 631 so as to obtain a base layer image. The decoding unit633 decodes the non-base layer image coded stream extracted by thedemultiplexer 631, so as to obtain a non-base layer image.

The decoding device 50 (FIG. 12) is applicable to the decoding unit 632and the coding unit 633 of the layer image decoding device 630. In otherwords, in decoding of each layer, a decoded image can be converted intoa desired image with a different dynamic range. In addition, thedecoding unit 612 and the decoding unit 613 can perform coding (that is,a flag or a parameter can be shared) by using the mutually same flags orparameters (for example, a syntax element or the like regarding aprocess of images), and thus it is possible to minimize a reduction incoding efficiency.

[Eighth Embodiment]

(Configuration Example of Television Apparatus)

FIG. 84 exemplifies a television apparatus to which the presenttechnology is applied. The television apparatus 900 includes an antenna901, a tuner 902, a demultiplexer 903, a decoder 904, a video signalprocessing unit 905, a display unit 906, an audio signal processing unit907, a speaker 908, and an external interface unit 909. In addition, thetelevision apparatus 900 includes a control unit 910, a user interface911, and the like.

The tuner 902 selects a desired channel from a broadcast signal which isreceived via the antenna 901, demodulates the selected channel, andoutputs a coded bit stream which is obtained through demodulation, tothe demultiplexer 903.

The demultiplexer 903 extracts a video or an audio packet of a programwhich is a viewing target from the coded bit stream, and outputs thedata on the extracted packet to the decoder 904. In addition, thedemultiplexer 903 supplies a packet of data such as electronic programguide (EPG) to the control unit 910. Further, the demultiplexer or thelike may perform descrambing when the coded stream is scrambled.

The decoder 904 decodes the packet, and outputs video data and audiodata generated through the decoding to the video signal processing unit905 and the audio signal, processing unit 907, respectively.

The video signal processing unit 905 performs noise removal or a videoprocess or the like in accordance with user's settings on the videodata. The video signal processing unit 905 generates video data of aprogram which is displayed on the display unit 906, or image data or thelike through a process based on an application which is supplied via anetwork. In addition, the video signal processing unit 905 generatesvideo data for displaying a menu screen such as selection of items, andsuperimposes the video data on the video data of a program. The videosignal processing unit 905 generates a driving signal on the basis ofthe video data generated in this way, so as to generate the display unit906.

The display unit 906 drives a display device (for example, a liquidcrystal display element) on the basis of the driving signal from thevideo signal processing unit 905 so as to display a video of a programor the like.

The audio signal processing unit 907 performs a process such as noiseremoval on the audio data, and performs D/A conversion or amplificationon the processed audio data which is then supplied to the speaker 908,thereby outputting sounds.

The external interface unit 909 is an interface for connection to anexternal apparatus or the network, and transmits and receives data suchas video data or audio data.

The control unit 901 is connected to the user interface unit 911. Theuser interface unit 911 is constituted by an operation switch, a remotecontrol signal reception portion, and the like, and supplies anoperation signal corresponding to a user's operation to the control unit910.

The control unit 910 is formed by using a central processing unit (CPU),memories, and the like. The memories store a program executed by theCPU, a variety of data which is necessary in the CPU performing aprocess, EPG data, data acquired via the network, and the like. Theprogram stored in the memories is read and executed by the CPU, forexample, when the television apparatus 900 is started. The CPU executesthe program, and thus controls each unit so that the televisionapparatus 900 performs an operation responding to a user's operation.

In addition, the television apparatus 900 is provided with a bus 912which connects the tuner 902, the demultiplexer 903, the video signalprocessing unit 905, the audio signal processing unit 907, the externalinterface unit 909, and the control unit 910, to each other.

In the television apparatus having the configuration, a function of thedecoding device (decoding method) of the present application is providedin the decoder 904. For this reason, it is possible to convert a decodedimage into a desired image with a different dynamic range.

[Ninth Embodiment]

(Configuration Example of Mobile Phone)

FIG. 35 exemplifies a schematic configuration of a mobile phone to whichthe present disclosure is applied. The mobile phone 920 includes acommunication unit 922, an audio codec 923, a camera unit 926, an imageprocessing unit 927, a multiplexer/demultiplexer 928, arecording/reproducing unit 929, a display unit 930, and a control unit931. Theses constituent elements are connected to each other via a bus933.

In addition, the communication unit 922 is connected to an antenna 921,and the audio codec 923 is connected to a speaker 924 and a microphone925. Further, the control unit 931 is connected to an operation unit932.

The mobile phone 920 performs various operations such as transmissionand reception of audio signals, transmission and reception of electronicmails or image data, capturing of an image, and recording of data invarious operation modes such as a speech mode and a data communicationmode.

In the speech mode, an audio signal generated by the microphone 925undergoes conversion into audio data or data compression in the audiocodec 923, and is then supplied to the communication unit 922. Thecommunication unit 922 performs a modulation process or a frequencyconversion process on the audio data so as to generate a transmissionsignal. Further, the communication unit 922 transmits the transmissionsignal to the antenna 921 so as to transmit the transmission signal to abase station (not illustrated). Furthermore, the communication unit 922performs amplification, a frequency conversion process, and ademodulation process on a signal which is received via the antenna 921,and supplies the generated audio data to the audio codec 923. The audiocodec 923 performs data decompression on the audio data or converts theaudio data into an analog audio signal, and outputs the generated audiosignal to the speaker 924.

Further, in the data communication mode, in a case of transmitting amail, the control unit 931 receives text data which is input by usingthe operation unit. 932, and displays the input text on the display unit930. Moreover, the control unit 931 generates mail data in response toan instruction made by the user by using the operation unit 932, andsupplies the generated mail data to the communication unit 922. Thecommunication unit 922 performs a modulation process or a frequencyconversion process on the mail data, and transmits the generatedtransmission signal from the antenna 921. Further, the communicationunit 922 performs amplification, a frequency conversion process, and ademodulation process on a signal which is received via the antenna 921,so as to recover mail data. The mail data is supplied to the displayunit 930, and thus content of the mail is displayed.

In addition, the mobile phone 920 may store the received mail data on arecording medium by using the recording/reproducing unit 929. Therecording medium is any rewritable recording medium. For example, therecording medium is a semiconductor memory such as a RAM or a built-inflash memory, or a removable medium such as a hard disk, a magneticdisk, a magneto-optical disc, an optical disc, a universal serial bus(USB) memory, or a memory card.

In a case where image data is transmitted in the data communicationmode, image data generated by the camera unit 926 is supplied to theimage processing unit 927. The image processing unit 927 performs acoding process on the image data so as to generate coded data.

Further, the multiplexer/demultiplexer 928 multiplexes the image streamwhich has been generated by the image processing unit 927 and the audiodata which is supplied from the audio codec 923, and supplies themultiplexed data to the communication unit 922. The communication unit922 performs a modulation process or a frequency conversion process onthe multiplexed data, and transmits an obtained transmission signal tothe antenna 921. Furthermore, the communication unit 922 performs anamplification process, a frequency conversion process, and ademodulation process on a signal which is received via the antenna 921so as to recover multiplexed data. The multiplexed data is supplied tothe multiplexer/demultiplexer 928. The multiplexer/demultiplexer 928demultiplexes the multiplexed data, and supplies coded data to the imageprocessing unit 927 and audio data to the audio codec 923. The imageprocessing unit 927 decodes the coded data so as to generate image data.The image data is supplied to the display unit 930 so as to allow thereceived image to be displayed. The audio codec 923 converts the audiodata into an analog audio signal which isthen supplied to the speaker924 so as to output a received sound.

In the mobile phone apparatus having the configuration, functions of thecoding device and the decoding device (the coding method and thedecoding method) of the present application are provided in the imageprocessing unit 927. For this reason, an image can be coded so that adecoded image can be converted into a desired image with a differentdynamic range during decoding. In addition, it is possible to convert adecoded image into a desired image with a different dynamic range.

[Tenth Embodiment]

(Configuration Example of Recording/Reproducing Apparatus)

FIG. 86 exemplifies a schematic configuration of a recording/reproducingapparatus to which the present technology is applied. Therecording/reproducing apparatus 940 records, for example, audio data andvideo data of a received broadcast program on a recording medium, andprovides the recorded data to a user at a timing responding to aninstruction from the user. In addition, the recording/reproducingapparatus 940 may acquire, for example, audio data and image data fromother apparatuses, and may record the data on the recording medium.Further, the recording/reproducing apparatus 940 codes and outputs theaudio data or video data recorded on the recording medium so that imagedisplay or sound output can be performed in a monitor device.

The recording/reproducing apparatus 940 includes a tuner 941, anexternal interface unit 942, an encoder 943, a hard disk drive (HDD)unit 944, a disc drive 945, a selector 946, a decoder 947, an on-screendisplay (OSD) unit 948, a control unit 949, and a user interface unit950.

The tuner 941 selects a desired channel from a broadcast signal which isreceived via an antenna (not illustrated). In addition, the tuner 941outputs a coded bit stream which is obtained by demodulating thereceived signal of the desired channel, to the selector 946.

The external interface unit 942 includes any one of an IEEE1394interface, a network interface, a USB interface, a flash memoryinterface, or the like. The external interface unit 942 is an interfacewhich is connected to an external apparatus, a network, a memory card,or the like, and receives data such as video data or audio data to berecorded.

The encoder 943 codes vide data or audio data in a predetermined methodin a case where the video data and the audio data supplied from theexternal interface unit 942 are not coded, and outputs a coded bitstream to the selector 946.

The HDD unit 944 records content data such as a video and a sound,various programs, and other data in a built-in hard disk, and reads thedata from the hard disk when the video and the sound are reproduced.

The disc drive 945 records and reproduces data on and from an opticaldisc which is installed therein. The optical disc may be, for example, aDVD disc (DVD-Video, DVD-RAM, DVD-R, DVD-RW, DVD+R, DVD+RW, or thelike), a Blu-ray (registered trademark) disc, or the like.

When a video and a sound are recorded, the selector 946 selects a codedbit stream which is input from the tuner 941 or the encoder 943, andoutputs the selected coded bit stream to the HDD unit 944 or the discdrive 945. In addition, when a video and a sound are reproduced, theselector 946 outputs a coded bit stream which is output from the HDDunit 944 or the disc drive 945, to the decoder 947.

The decoder 947 decodes the coded bit stream. In addition, the decoder947 supplies video data generated through the decoding process, to theOSD unit 948. Further, the decoder 947 outputs audio data generatedthrough the decoding process.

The OSD unit 948 generates video data for displaying a menu screen suchas selection of items, and superimposes and outputs the video data onvideo data which is output from the decoder 947.

The control unit 949 is connected to the user interface unit 950. Theuser interface unit 950 is constituted by an operation switch, a remotecontrol signal reception portion, and the like, and supplies anoperation signal corresponding to a user's operation to the control unit949.

The control unit 949 is formed by using a central processing unit (CPU),memories, and the like. The memories store a program executed by theCPU, a variety of data which is necessary in the CPU performing aprocess, EPG data, data acquired via the network, and the like. Theprogram stored in the memories is read and executed by the CPU at apredetermined timing, for example, when the recording/reproducingapparatus 940 is started. The CPU executes the program, and thuscontrols each unit so that the recording/reproducing apparatus 940performs an operation responding to a user's operation.

In the recording/reproducing apparatus having the configuration, afunction of the decoding device (decoding method) of the presentapplication is provided in the decoder 947. For this reason, it ispossible to convert a decoded image into a desired image with adifferent dynamic range.

[Eleventh Embodiment]

(Configuration Example of Imaging Apparatus)

FIG. 87 exemplifies a schematic configuration of an imaging apparatus towhich the present technology is applied. The imaging apparatus 960captures an image of a subject, and displays the image of the subject ona display unit or records the image on a recording medium as image data.

The imaging apparatus 960 includes an optical block 961, an imaging unit962, a camera signal processing unit 963, an image data processing unit964, a display unit 965, an external interface unit 966, a memory unit967, a medium drive 968, an OSD unit 969, and a control unit 970. Inaddition, the control unit 970 is connected to a user interface 971.Further, the image data processing unit 964, the external interface unit966, the memory unit 967, the medium drive 968, the OSD unit 969, thecontrol unit 970, and the like are connected to each other via a bus972.

The optical block 961 includes a focus lens, a diaphragm mechanism, andthe like. The optical block 961 forms an optical image of a subject onan imaging surface of the imaging unit 962. The imaging unit 962includes an image sensor such as a CCD or a CMOS, and generates anelectrical signal corresponding to the optical image throughphotoelectric conversion, and supplies the electrical signal to thecamera signal processing unit 963.

The camera signal processing unit 963 performs various camera signalprocesses such as knee correction, gamma correction, and colorcorrection, on the image signal which is input from the imaging unit962. The camera signal processing unit 963 supplies the image datahaving undergone the camera signal processes to the image dataprocessing unit 964.

The image data processing unit 964 codes the image data which issupplied from the camera signal processing unit 963. The image dataprocessing unit 964 supplies coded data generated through the codingprocess to the external interface unit 966 or the medium drive 968.Further, the image data processing unit 964 decodes the coded data whichis supplied from the external interface unit 966 or the medium drive968. Furthermore, the image data processing unit 964 supplies image datagenerated through the decoding process to the display unit 965.Moreover, the image data processing unit 964 supplies image data whichis supplied from the camera signal processing unit 963, to the displayunit 965, or superimposes display data which is acquired from the OSDunit 969, on image data which is then output to the display unit 965.

The OSD unit 969 generates and outputs display data such as a menuscreen formed by symbols, characters, or figures, or an icon, to theimage data processing unit 964.

The external interface unit 966 is formed by, for example, a USB inputand output terminal, and is connected to a printer when an image isprinted. In addition, the external interface unit 966 is connected to adrive as necessary. A removable medium such as a magnetic disk or anoptical disc is installed in the drive as appropriate, and a computerprogram read from the removable medium is installed therein asnecessary. Further, the external interface unit 966 includes a networkinterface which is connected to a predetermined network such as a LAN orthe Internet. The control unit 970 may read coded data from the mediumdrive 968, for example, in response to an instruction from the userinterface 971, and may supply the coded data to other apparatuses whichis connected thereto via the network, from the external interface unit966. In addition, the control unit 970 may acquire coded data or imagewhich is supplied from other apparatuses via the network, through theexternal interface unit 966, and may supply the data to the image dataprocessing unit 964.

A recording medium driven by the medium drive 968 may be any readableand writable removable medium such as a magnetic disk, a magneto-opticaldisc, an optical disc, or a semiconductor memory. In addition, therecording medium may be any kind of removable medium, may be a tapedevice, may be a disk, and may be a memory card. Of course, a noncontactintegrated circuit (IC) card or the like may be used.

Further, the medium drive and a recording medium may be integrallyformed so as to be constituted by a non-portable storage unit such as abuilt-in hard disk drive or a solid state drive (SSD).

The control unit 970 is formed by using a CPU. The memory unit 967stores a program executed by the control unit 970, a variety of datawhich is necessary in the control unit 970 performing a process, and thelike. The program stored in the memory unit 967 is read and executed bythe control unit 970, a predetermined timing, for example, when theimaging apparatus 960 is started. The control unit 970 executes theprogram, and thus controls each unit so that the imaging apparatus 960performs an operation responding to a user's operation.

In the imaging apparatus having the configuration, functions of thecoding device and the decoding device (the coding method and thedecoding method) of the present application are provided in the imagedata processing unit 964. For this reason, an image can be coded so thata decoded image can be converted into a desired image with a differentdynamic range during decoding. In addition, it is possible to convert adecoded image into a desired image with a different dynamic range.

(Application Examples of Scalable Coding)

[First System]

Next, description will be made of a specific example of using scalablecoded (layer coded) data which is scalably coded. The scalable coding isused, for example, to select data to be transmitted as in an exampleillustrated in FIG. 88.

In a data transmission system 1000 illustrated in FIG. 88, a deliveryserver 1002 reads the scalable coded data stored in the scalable codeddata storage unit 1001, and delivers the scalable coded data to terminalapparatuses such as a personal computer 1004, an AV apparatus 1005, atablet device 1006, and a mobile phone 1007 via a network 1003.

At this time, the delivery server 1002 selects and transmits coded datawith appropriate quality on the basis of performances of the terminalapparatuses, communication circumstances, or the like. If the deliveryserver 1002 unnecessarily transmits high quality data, it is unable tobe said that a high quality image is obtained in the terminal apparatus,and there is a concern that delay or overflow may occur. In addition,there is a concern that high quality data may unnecessarily occupy acommunication band, and may unnecessarily increase a load on theterminal apparatus. Conversely, if the delivery server 1002unnecessarily transmits low quality data, there is a concern that animage with sufficient image quality may not be obtained in the terminalapparatus. For this reason, the delivery server 1002 reads and transmitscoded data with quality (layer) which is suitable for the performancesof the terminal apparatuses or the communication circumstances from thescalable coded data storage unit 1001.

Here, it is assumed that the scalable coded data storage unit 1001stores scalable coded data (BL+EL) 1011 which is scalably coded. Thescalable coded data (BL+EL) 1011 is coded data including both a baselayer and an enhancement layer, and is data which allows both a baselayer image and an enhancement layer image to be obtained throughdecoding.

The delivery server 1002 selects an appropriate layer on the basis of aperformance of a terminal apparatus to which data is transmitted orcommunication circumstances, and reads data of the layer. For example,the delivery server 1002 reads the scalable coded data (BL+EL) 1011which has high quality from the scalable coded data storage unit 1001,and transmits the data as it is, in relation to the personal computer1004 or the tablet device 1006 having a high processing performance. Incontrast, for example, in relation to the AV apparatus 1005 or themobile phone 1007 having a low processing performance, the deliveryserver 1002 extracts base layer data from the scalable coded data(BL+EL) 1011, and transmits the data as scalable coded data (BL) 1012which is the same content data as the scalable coded data (BL+EL) 1011in terms of content but has lower quality than the scalable coded data(BL+EL) 1011.

As mentioned above, since a data amount can be easily adjusted by usingthe scalable coded data, it is possible to minimize the occurrence ofdelay or overflow or to minimize an unnecessary increase in a load on aterminal apparatus or a communication medium. In addition, redundancybetween layers is reduced in the scalable coded data (BL+EL) 1011, andthus a data amount thereof can be further reduced than in a case wherecoded data of each layer is used as separate data. Therefore, a storageregion of the scalable coded data storage unit 1001 can be used moreefficiently.

In addition, various apparatuses such as the personal computer 1004 tothe mobile phone 1007 can be employed as terminal apparatuses and thusperformances of hardware of the terminal apparatuses are differentdepending on the apparatuses. Further, there are various applicationswhich are executed by the terminal apparatuses, and thus there are alsovarious performances of software thereof. Furthermore, all communicationline networks including a wired network, a wireless network, or bothnetworks such as, for example, the Internet or a local area network(LAN) can be employed as the network 1003 which is a communicationmedium, and there are various data transmission performances. Moreover,there is a concern that a data transmission performance may varydepending on other communication circumstances or the like.

Therefore, before starting data transmission, the delivery server 1002may perform communication with a terminal apparatus which is atransmission destination of the data, so as to obtain informationregarding performances of the terminal apparatus such as a hardwareperformance of the terminal apparatus and a performance of anapplication (software) executed by the terminal apparatus, andinformation regarding communication circumstances such as an availablebandwidth of the network 1003. In addition, the delivery server 1002 mayselect an appropriate layer on the basis of the information obtainedhere.

Further, the extraction of a layer may be performed by the terminalapparatus. For example, the personal computer 1004 may decode thetransmitted scalable coded data (BL+EL) 1011 so as to display a baselayer image and display an enhancement layer image. Furthermore, forexample, the personal computer 1004 may extract the base layer scalablecoded data (BL) 1012 from the transmitted scalable coded data (BL+EL)1011 so as to store the data, to transmit the data to other devices, orto decode the data for display of a base layer image.

Of course, the number of scalable coded data storage units 1001, thenumber of delivery servers 1002, the number of networks 1003, and thenumber of terminal apparatuses are all arbitrary. In addition, in theabove description, a description has been made of an example in whichthe delivery server 1002 transmits data to the terminal apparatus, but ausage example is not limited thereto. The data transmission system 1000is applicable to any system as long as the system selects and transmitsan appropriate layer on the basis of a performance of a terminalapparatus, communication circumstances, or the like when coded datawhich is scalably coded is transmitted to the terminal apparatus.

[Second System]

The scalable coding is used, for example, for transmission using aplurality of communication media as in an example illustrated in FIG.89.

In a data transmission system 1100 illustrated in FIG. 89, abroadcasting station 1101 transmits base layer scalable coded data (BL)1121 by using a terrestrial broadcast 1111. In addition, thebroadcasting station 1101 transmits (for example, packetizes andtransmits) enhancement layer scalable coded data (EL) 1122 via anynetwork 1112 formed by a wired network, a wireless network, or bothnetworks.

A terminal apparatus 1102 has a reception function of the terrestrialbroadcast 1111 which is broadcasted by the broadcasting station 1101,and receives the base layer scalable coded data (BL) 1121 which istransmitted via the terrestrial broadcast 1111. In addition, theterminal apparatus 1102 further has a communication function ofperforming communication using the network 1112, and receives theenhancement layer scalable coded data (EL) 1122 which is transmitted viathe network 1112.

The terminal apparatus 1102 may decode the base layer scalable codeddata (BL) 1121 which is acquired via the terrestrial broadcast 1111, forexample, in response to an instruction from a user, so as to obtain abase layer image, to store the image, and to transmit the image to otherapparatuses.

In addition, for example, in response to an instruction from a user, theterminal apparatus 1102 may combine the base layer scalable coded data(BL) 1121 which is acquired via the terrestrial broadcast 1111 with thenon-base layer scalable coded data (EL) 1122 which is acquired via thenetwork 1112 so as to obtain scalable coded data (BL+EL), and may decodethe data so as to obtain a base layer image, to store the image, and totransmit the image to other apparatuses.

As mentioned above, the scalable coded data may be transmitted via acommunication medium which is different for each layer, for example. Inthis case, a load can be distributed, and thus it is possible tominimize the occurrence of delay or overflow.

In addition, a communication medium used for transmission may beselected for each layer depending on circumstances. For example, thebase layer scalable coded data (BL) 1121 having a relatively largeamount of data may be transmitted via a communication media having awide bandwidth, and the enhancement layer scalable coded data (EL) 1122having a relatively small amount of data may be transmitted via acommunication medium having a narrow bandwidth. In addition, forexample, a communication medium for transmitting the enhancement layerscalable coded data (EL) 1122 may be changed between the network 1112and the terrestrial broadcast 1111 depending on an available bandwidthof the network 1112. Of course, this is also the same for data of anylayer.

The control is performed as mentioned above, and thus it is possible tofurther minimize an increase in a load in data transmission.

Of course, the number of layers is arbitrary, and the number ofcommunication media used for transmission is also arbitrary. Further,the number of terminal apparatuses 1102 serving as a data transmissiondestination is also arbitrary. Furthermore, in the above description,the description has been made of broadcasting from the broadcastingstation 1101 as an example, but a usage example is not limited thereto.The data transmission system 1100 is applicable to any system as long asthe system splits coded data which is scalably coded into a plurality ofdata items in the unit of layers and transmits the data items via aplurality of lines.

[Third System]

The scalable coding is used, for example, to store coded data as in anexample illustrated in FIG. 90.

In an imaging system 1200 illustrated in FIG. 90, an imaging apparatus1201 scalably codes image data which is obtained by imaging a subject1211, and supplies resultant data to a scalable coded data storagedevice 1202 as scalable coded data (BL+EL) 1221.

The scalable coded data storage device 1202 stores the scalable codeddata (BL+EL) 1221 which is supplied from the imaging apparatus 1201,with quality based on circumstances. For example, in a case of thenormal, time, the scalable coded data storage device 1202 extracts baselayer data from the scalable coded data (BL+EL) 1221, and stores thedata as base layer scalable coded data (BL) 1222 having a small amountof data with low quality. In contrast, for example, in a case of thenotice time, the scalable coded data storage device 1202 stores thescalable coded data (BL+EL) 1221 having a large amount of data with highquality as it is.

Accordingly, since the scalable coded data storage device 1202 canpreserve a high quality image as necessary only, it is possible tominimize an increase in a data amount while minimizing a reduction inthe value of an image due to image quality deterioration, and thus toimprove use efficiency of a storage region.

For example, the imaging apparatus 1201 is assumed to be a monitoringcamera. In a case (a case of the normal time) where a monitoring target(for example, an trespasser) is not reflected in a captured image, thereis a high probability that content of the captured image may not beimportant, and thus a reduction in a data amount is prioritized, and theimage data (scalable coded data) is stored with low quality. Incontrast, in a case (a case of the notice time) where a monitoringtarget is reflected in a captured image as the subject 1211, there is ahigh probability that content of the captured image may be important,and thus image quality is prioritized, and the image data (scalablecoded data) is stored with high quality.

In addition, the normal time and the notice time may be determined, forexample, by the scalable coded data storage device 1202 analyzing animage. Further, the normal time and the notice time may be determined,for example, by the imaging apparatus 1201, and a determination resultmay be transmitted to the scalable coded data storage device 1202.

In addition, a determination criterion of the normal time and the noticetime is arbitrary, and content of a captured image which is used as adetermination criterion is arbitrary. Of course, conditions other thanthe content of a captured image may be used as a determinationcriterion. For example, the normal time and the notice time may bechanged on the basis of the magnitude, a waveform, or the like of arecorded sound, and may be changed, for example, for each predeterminedtime interval, or by an external instruction such as an instruction froma user.

In addition, in the above description, an example of changing two statesincluding the normal time and the notice time has been described, butthe number of states is arbitrary, and, for example, three or morestates such as the normal time, the slight notice time, the notice time,the great notice time, may be changed. Here, an upper limit number ofchanged states depends on the number of layers of scalable coded data.

In addition, the imaging apparatus 1201 may determine the number ofscalable coded layers on the basis of a state. For example, in a case ofthe normal time, the imaging apparatus 1201 may generate the base layerscalable coded data (BL) 1222 having a small amount of data with lowquality, and may supply the data to the scalable coded data storagedevice 1202. Further, for example, in a case of the notice time, theimaging apparatus 1201 may generate the base layer and non-base layerscalable coded data (BL+EL) 1221 having a large amount of data with highquality, and may supply the data to the scalable coded data storagedevice 1202.

In the above description, the description has been made of themonitoring camera as an example, but usage of the imaging system 1200 isarbitrary and is not limited to a monitoring camera.

[Twelfth Embodiment]

(Other Examples)

In the above description, examples of apparatuses or systems to whichthe present technology is applied have been described, but the presenttechnology is not limited thereto, and may be realized by allconfigurations mounted in a device forming the apparatus or the system,for example, a processor as system large scale integration (LSI) or thelike, a module using a plurality of processors, a unit using a pluralityof modules, a set in which other functions are added to the unit, andthe like (a configuration of a part of an apparatus).

(Configuration Example of Video Set)

With reference to FIG. 91, description will be made of an example inwhich the present technology is realized by a set. FIG. 91 illustratesan example of a schematic configuration of a video set to which thepresent technology is applied.

Multi-functioning of an electronic apparatus has recently progressed,and thus there are many cases where, when a partial configuration issold or provided in development or manufacturing thereof, not only aconfiguration having a single function is realized but also a set havinga plurality of functions through combination of a plurality ofconfigurations having related functions is implemented.

A video set 1300 illustrated in FIG. 91 has a multi-functionalconfiguration, and is one in which a device having a function regardingcoding or decoding (one or both of the coding or decoding may be used)of an image is combined with a device having other functions related tothe function.

As illustrated in FIG. 91, the video set 1300 includes a module groupsuch as a video module 1311, an external memory 1312, a power managementmodule 1313, and a front end module 1314, and devices having relatedfunctions, such as a connectivity 1321, a camera 1322, and a sensor1323.

The module is a component having a unified function by collectingseveral mutually related component functions. A specific physicalconfiguration of the module is arbitrary, and, for example, a pluralityof processors having each function, electronic circuit elements such asresistors and capacitors, other devices, and the like may be disposed ona wiring board and integrally formed. In addition, a module may becombined with other modules, processors, or the like, so as to form anew module.

In a case of the example of FIG. 91, the video module 1311 is acombination of configurations having functions regarding imageprocessing, and includes an application processor, a video processor, abroadband modem 1333, and an RF module 1334.

The processor is one in which configurations having predeterminedfunctions are integrated into a semiconductor chip by using a system ona chip (SoC), and there may be a processor which is called, for example,system large scale integration (LSI). The configurations havingpredetermined functions may be logic circuits (hardware configuration),may be a CPU, a ROM, a RAM, and the like, and programs (softwareconfiguration) executed by using the configurations, and may be acombination of both thereof. For example, the process includes a logiccircuit, a CPU, a ROM, a RAM, and the like, some functions may berealized by the logic circuit (hardware configuration), and otherfunctions may be realized by the program (software configuration)executed by the CPU.

The application processor 1331 of FIG. 91 is a processor which executesan application related to image processing. The application executed bythe application processor 1331 may perform a calculation process inorder to realize a predetermined function, and may also controlconstituent elements inside and outside the video module 1311, such asthe video processor 1332.

The video processor 1332 is a processor having a function related tocoding/decoding (one or both thereof) of an image.

The broadband modem 1333 is a processor (or a module) which performs aprocess related to wired or wireless (or both thereof) broadbandcommunication which is performed via a broadband line such as theInternet or a public telephone line. For example, the broadband modem1333 digitally modulates data (digital signal) to be transmitted, forconversion into an analog signal, or demodulates a received analogsignal for conversion into data (digital signal). For example, thebroadband modem 1333 can digitally modulate/demodulate any informationsuch as image data processed by the video processor 1332, a stream inwhich the image data is coded, an application program, or setting data.

The RF module 1334 is a module which performs frequency conversion,modulation/demodulation, amplification, filtering, and the like on aradio frequency (RF) signal which is transmitted and received via anantenna. For example, the RF module 1334 performs frequency conversionor the like on a baseband signal generated by the broadband modem 1333so as to generate an RF signal. In addition, for example, the RF module1334 performs frequency conversion or the like on an RF signal which isreceived via the front end module 1314, so as to generate a basebandsignal.

Further, in FIG. 91, as indicated by a dotted line 1341, the applicationprocessor 1331 and the video processor 1332 may be integrally formed soas to configure a single processor.

The external memory 1312 is a module which is provided outside the videomodule 1311 and includes a storage device used by the video module 1311.The storage device of the external memory 1312 may be implemented by anyphysical configuration, but is generally used to store a large volume ofdata such as image data of frame units, and is thus preferablyimplemented by a large capacity semiconductor memory which is relativelycheap, such as a dynamic random access memory (DRAM).

The power management module 1313 manages and control power which issupplied to the video module 1311 (each constituent element in the videomodule 1311).

The front end module 1314 is a module which provides a front endfunction (a circuit of a transmission and reception end of an antennaside) to the RF module 1334. As illustrated in FIG. 91, the front endmodule 1314 includes, for example, an antenna portion 1351, a filter1352, and an amplification portion 1353.

The antenna portion 1351 includes an antenna and peripheral constituentelements which transmit and receive a wireless signal. The antennaportion 1351 transmits a signal which is supplied from the amplificationportion 1353 as a wireless signal, and supplies the received wirelesssignal to the filter 1352 as an electrical signal (RF signal). Thefilter 1352 performs a filter process on the received RF signal which isreceived via the antenna portion 1351, and supplies a processed RFsignal to the RF module 1334. The amplification portion 1353 amplifiesthe RF signal supplied from the RF module 1334, and supplies theamplified signal to the antenna portion 1351.

The connectivity 1321 is a module having a function related toconnection to an external device. A physical configuration of theconnectivity 1321 is arbitrary. For example, the connectivity 1321includes a constituent element having a communication function otherthan a communication standard supported by the broadband modem 1333, anexternal input and output terminal, and the like.

For example, the connectivity 1321 may include a module having acommunication function conforming to a wireless communication standardsuch as Bluetooth (registered trademark) or IEEE 802.11 (for example,Wireless Fidelity (Wi-Fi, registered trademark), near fieldcommunication (NFC), or Infrared Data Association (IrDA)), an antennawhich transmits and receives a signal conforming to the standard, andthe like. In addition, for example, the connectivity 1321 may include amodule having a communication function conforming to a wiredcommunication standard such as Universal Serial Bus (USB) orHigh-Definition Multimedia Interface (HDMI) (registered trademark), or aterminal conforming to the standard. Further, for example, theconnectivity 1321 may have other data (signal) transmission functions inan analog input and output terminal or the like.

In addition, the connectivity 1321 may include a device of atransmission destination of data (signal). For example, the connectivity1321 may include a drive (including not only a removable medium drivebut also a hard disk, a solid state drive (SSD), and a network attachedstorage (NSA)) which performs reading or writing of data from or to arecording medium such as a magnetic disk, an optical disc, amagneto-optical disc, or a semiconductor memory. Further, theconnectivity 1321 may include an image or sound output device (amonitor, a speaker, or the like).

The camera 1322 is a module having a function of capturing an image of asubject, and acquiring image data of the subject. The image dataacquired by the camera 1322 capturing an image of the subject issupplied to, for example, the video processor 1332, and is coded.

The sensor 1323 is a module having any sensor function, such as an audiosensor, an ultrasonic sensor, an optical sensor, an illuminance sensor,an infrared sensor, an image sensor, a rotation sensor, an angle sensor,an angular velocity sensor, a speed sensor, an acceleration sensor, atilt sensor, a magnetic identification sensor, an impact sensor, or atemperature sensor. Data detected by the sensor 1323 is supplied to, forexample, the application processor 1331, and is used by an applicationor the like.

In the above description, a configuration described as a module may berealized as a processor, and, conversely, a configuration described as aprocessor may be realized as a module.

In the video set 1300 having the above-described configuration, thepresent disclosure is applicable to the video processor 1332 asdescribed later. Therefore, the video set 1300 may be implemented as aset to which the present technology is applied.

(Configuration Example of Video Processor)

FIG. 92 illustrates an example of a schematic configuration of the videoprocessor 1332 (FIG. 91) to which the present technology is applied.

In a case of the example of FIG. 92, the video processor 1332 has afunction of receiving a video signal and an audio signal and coding thesignals in a predetermined method, and a function of decoding codedvideo data and audio data so as to reproduce a video signal and an audiosignal.

As illustrated in FIG. 92, the video processor 1332 includes a videoinput processing portion 1401, a first image enlargement/reductionportion 1402, a second image enlargement/reduction portion 1403, a videooutput processing portion 1404, a frame memory 1405, and a memorycontrol portion 1406. In addition, the video processor 1332 includes anencode/decode engine 1407, video elementary stream (ES) buffers 1408Aand 1408B, and audio ES buffers 1409A and 1409B. Further, the videoprocessor 1332 includes an audio encoder 1410, an audio decoder 1411, amultiplexer (MUX) 1412, a demultiplexer (DMUX) 1413, and a stream buffer1414.

The video input processing portion 1401 acquires a video signal which isinput from, for example, the connectivity 1321 (FIG. 91) or the like,and converts the video signal into digital image data. The first imageenlargement/reduction portion 1402 performs format conversion or animage enlargement or reduction process on the image data. The secondimage enlargement/reduction portion 1403 performs an image enlargementor reduction process in accordance with a format at a destination of avideo which is output via the video output processing portion 1404 onthe image data, or performs the same format conversion or imageenlargement or reduction process as in the first imageenlargement/reduction portion 1402 on the image data. The video outputprocessing portion 1404 performs format conversion, conversion into ananalog signal, or the like on the image data, and outputs a convertedsignal to, for example, the connectivity 1321 (FIG. 91) or the like as areproduced video signal.

The frame memory 1405 is a memory for image data, shared by the videoinput processing portion 1401, the first image enlargement/reductionportion 1402, the second image enlargement/reduction portion 1403, thevideo output processing portion 1404, and the encode/decode engine 1407.The frame memory 1405 is implemented by a semiconductor memory such as aDRAM.

The memory control portion 1406 receives a synchronization signal fromthe encode/decode engine 1407, and controls writing/reading access tothe frame memory 1405 according to a schedule for access to the framememory 1405, written in an access management table 1406A. The accessmanagement table 1406A is updated by the memory control portion 1406 inaccordance with processes performed by the encode/decode engine 1407,the first image enlargement/reduction portion 1402, the second imageenlargement/reduction portion 1403, and the like.

The encode/decode engine 1407 performs an encode process on image data,and a decode process on a video stream which is coded data of imagedata. For example, the encode/decode engine 1407 codes image data readfrom the frame memory 1405, and sequentially writes the coded image datato the video ES buffer 1408A as a video stream. In addition, forexample, video streams are sequentially read from the video ES buffer1408B so as to be decoded, and are sequentially written to the framememory 1405 as image data. The encode/decode engine 1407 uses the framememory 1405 as a work area in the coding or decoding. Further, theencode/decode engine 1407 outputs an synchronization signal to thememory control portion 1406, for example, at a timing of starting aprocess on each macroblock.

The video ES buffer 1408A buffers a video stream generated by theencode/decode engine 1407, and supplies the buffered video stream to themultiplexer (MUX) 1412. The video ES buffer 1408B buffers a video streamsupplied from the demultipiexer (DMUX) 1413, and supplies the bufferedvideo stream to the encode/decode engine 1407.

The audio ES buffer 1409A buffers an audio stream generated by the audioencoder 1410, and supplies the buffered audio stream to the multiplexer(MUX) 1412. The audio ES buffer 1409B buffers an audio stream suppliedfrom the demultiplexer (DMUX) 1413, and supplies the buffered audiostream to the audio decoder 1411.

The audio encoder 1410, for example, digitally converts an audio signalwhich is input from, for example, the connectivity 1321 (FIG. 91) or thelike, and codes the converted audio signal in a predetermined methodsuch as an MPEG audio method or AudioCode number 3 (AC3). The audioencoder 1410 sequentially writes an audio stream which is coded data ofthe audio signal to the audio ES buffer 1409A. The audio decoder 1411decodes an audio stream supplied from the audio ES buffer 1409B so as toperform conversion into an analog signal, or the like, and supplies theanalog signal to, for example, the connectivity 1321 (FIG. 91) or thelike as a reproduced audio signal.

The multiplexer (MUX) 1412 multiplexes the video stream and the audiostream. A method of the multiplexing (that is, a format of a bit streamgenerated through the multiplexing) is arbitrary. In addition, duringthe multiplexing, the multiplexer (MUX) 1412 may add predeterminedheader information to a bit stream. In other words, the multiplexer(MUX) 1412 can convert a format of the stream through the multiplexing.For example, the multiplexer (MUX) 1412 multiplexes the video stream andthe audio stream so as to perform conversion into a transport streamwhich is a bit stream with a transmission format. Further, for example,the multiplexer (MUX) 1412 multiplexes the video stream and the audiostream so as to perform conversion into data (file data) with arecording file format.

The demultiplexer (DMUX) 1413 demultiplexes a bit stream into which avideo stream and an audio stream are multiplexed, in a methodcorresponding to the multiplexing by the multiplexer (MUX) 1412. Inother words, the demultiplexer (DMUX) 1413 extracts a video stream andan audio stream from a bit stream which is read from the stream buffer1414 (separates the video stream and the audio stream therefrom). Thatis, the demultiplexer (DMUX) 1413 can convert a format of the streamthrough the demultiplexing (inverse conversion of the conversion in themultiplexer (MUX) 1412). For example, the demultiplexer (DMUX) 1413 mayacquire a transport stream which is supplied from, for example, theconnectivity 1321 or the broadband modem 1333 (FIG. 91), via the streambuffer 1414, and demultiplexes the transport stream so as to performconversion into a video stream and an audio stream. In addition, forexample, the demultiplexer (DMUX) 1413 may acquire file data which isread from various recording media by, for example, the connectivity 1321(FIG. 91), via the stream buffer 1414, and demultiplexes the transportstream so as to perform conversion into a video stream and an audiostream.

The stream buffer 1414 buffers a bit stream. For example, the streambuffer 1414 buffers a transport stream supplied from the multiplexer(MUX) 1412, and supplies the buffered transport stream to, for example,the connectivity 1321 or the broadband modem 1333 (FIG. 91) at apredetermined timing, or on the basis of a request or the like from anexternal device.

In addition, for example, the stream buffer 1414 buffers file datasupplied from the multiplexer (MUX) 1412, and supplies the buffered filedata to, for example, the connectivity 1321 (FIG. 91) so as to recordthe file data on various recording media at a predetermined timing, oron the basis of a request or the like from an external device.

Further, the stream buffer 1414 buffers a transport stream which isacquired via, for example, the connectivity 1321 or the broadband modem1333 (FIG. 91), and supplies the buffered transport stream to thedemultiplexer (DMUX) 1413 at a predetermined timing, or on the basis ofa request or the like from an external device.

Furthermore, the stream buffer 1414 buffers file data which is read fromvarious recording media in the connectivity 1321 (FIG. 91) or the like,and supplies the buffered transport stream to the demultiplexer (DMUX)1413 at a predetermined timing, or on the basis of a request or the likefrom an external device.

Next, an example of an operation of the video processor 1332 having theconfiguration will be described. For example, a video signal which isinput to the video processor 1332 from the connectivity 1321 (FIG. 91)or the like is converted into digital image data in a predeterminedscheme such as a 4:2:2 Y/Cb/Cr scheme by the video input processingportion 1401, and is sequentially written to the frame memory 1405. Thedigital image data is read to the first image enlargement/reductionportion 1402 or the second image enlargement/reduction portion 1403, andundergoes format conversion and an enlargement or reduction process in apredetermined scheme such as a 4:2:0 Y/Cb/Cr scheme so as to be writtento the frame memory 1405 again. The image data is coded by theencode/decode engine 1407 and is then written to the video ES buffer1408A as a video stream.

In addition, an audio signal which is input to the video processor 1332from the connectivity 1321 (FIG. 91) or the like is coded by the audioencoder 1410, and is written to the audio ES buffer 1409A an audiostream.

The video stream of the video ES buffer 1408A and the audio stream ofthe audio ES buffer 1409A are read to the multiplexer (MUX) 1412 so asto be multiplexed and be converted into a transport stream, file data,or the like. The transport stream generated by the multiplexer (MUX)1412 is buffered in the stream buffer 1414, and is then output to anexternal network via, for example, the connectivity 1321 or thebroadband modem 1333 (FIG. 91). In addition, the file data generated bythe multiplexer (MUX) 1412 is buffered in the stream buffer 1414, and isthen output to, for example, the connectivity 1321 (FIG. 91) so as to berecorded on various recording media.

Further, a transport stream which is input to the video processor 1332from an external network via, for example, the connectivity 1321 or thebroadband modem 1333 (FIG. 91) is buffered in the stream buffer 1414,and is then demultiplexed by the demultiplexer (DMUX) 1413. Furthermore,for example, file data which is read from various recording media, forexample, in the connectivity 1321 (FIG. 91) and is input to the videoprocessor 1332 is buffered in the stream buffer 1414, and is thendemultiplexed by the demultiplexer (DMUX) 1413. In other words, thetransport stream or the file data which is input to the video processor1332 is separated into a video stream and an audio stream by thedemultiplexer (DMUX) 1413.

The audio stream is supplied to the audio decoder 1411 via the audio ESbuffer 1409B so as to be decoded and to be reproduced as an audiosignal. In addition, the video stream which is written to the video ESbuffer 1408B is then sequentially read by the encode/decode engine 1407so as to be decoded and to be written to the frame memory 1405. Thedecoded image data undergoes an enlargement or reduction process in thesecond image enlargement/reduction portion 1403 so as to be written tothe frame memory 1405. Further, the decoded image data is read to thevideo output processing portion 1404 so as to undergo format conversionin a predetermined scheme such as a 4:2:2 Y/Cb/Cr scheme and further toundergo conversion into an analog signal, and thus a video signal isreproduced and output.

In a case where the present technology is applied to the video processor1332 having the configuration, the present disclosure related to eachembodiment described above may be applied to the encode/decode engine1407. In other words, for example, the encode/decode engine 1407 mayhave the function of the coding device or the decoding device related tothe first embodiment. Accordingly, the video processor 1332 can achievethe same effects as the effects described with reference to FIGS. 6 to13.

In addition, in the encode/decode engine 1407, the present technology(that is, the function of the image coding device or the image decodingdevice related to each embodiment described above) may be realized byhardware such as a logic circuit, may be realized by software such as anembedded program, and may be realized by both thereof.

(Another Configuration Example of Video Processor)

FIG. 93 illustrates another schematic configuration example of the videoprocessor 1332 (FIG. 91) to which the present technology is applied. Ina case of the example of FIG. 93, the video processor 1332 has afunction of coding and decoding video data in a predetermined method.

More specifically, as illustrated in FIG. 93, the video processor 1332includes a control portion 1511, a display interface 1512, a displayengine 1513, an image processing engine 1514, and an internal memory1515. In addition, the video processor 1332 includes a codec engine1516, a memory interface 1517, a multiplexer/demultiplexer (MUX DEMUX)1518, a network interface 1519, and a video interface 1520.

The control portion 1511 controls an operation of each processingportion of the video processor 1332, such as the display interface 1512,the display engine 1513, the image processing engine 1514, and the codecengine 1516.

As illustrated in FIG. 93, the control portion 1511 includes, forexample, a main CPU 1531, a sub-CPU 1532, and a system controller 1533.The main CPU 1531 executes a program or the like for controlling anoperation of each processing portion of the video processor 1332. Themain CPU 1531 generates a control signal according to the program or thelike, and supplies the control signal to each processing portion (thatis, controls an operation of each processing portion). The sub-CPU 1532assists the main CPU 1531. For example, the sub-CPU 1532 executes achild process, a sub-routine, or the like of a program executed by themain CPU 1531. The system controller 1533 controls operations of themain CPU 1531 and the sub-CPU 1532 by designating a program which is tobe executed by the main CPU 1531 and the sub-CPU 1532.

The display interface 1512 outputs image data to, for example, theconnectivity 1321 (FIG. 91) under the control of the control portion1511. For example, the display interface 1512 converts digital imagedata into an analog signal and outputs the analog signal to a monitordevice or the like of the connectivity 1321 (FIG. 91), or outputs thedigital image data to the monitor device as it is.

The display engine 1513 performs various conversion processes such asformat conversion, size conversion, and color gamut conversion on imagedata, so as to be suitable for a hardware specification of a monitordevice or the like which displays an image, under the control of thecontrol portion 1511.

The image processing engine 1514 performs a predetermined image processsuch as a filter process for improving image quality, on the image data,under the control of the control portion 1511.

The internal memory 1515 is a memory which is shared by the displayengine 1513, the image processing engine 1514, and the codec engine1516, and is provided in the video processor 1332. The internal memory1515 is used to transmit and receive data among, for example, thedisplay engine 1513, the image processing engine 1514, and the codecengine 1516. For example, the internal memory 1515 stores data suppliedfrom the display engine 1513, the image processing engine 1514, or thecodec engine 1516, and supplies the data to the display engine 1513, theimage processing engine 1514, or the codec engine 1516 as necessary (forexample, in response to a request). The internal memory 1515 may berealized by any storage device, but is generally often used to store asmall volume of data such as image data of the block unit or aparameter, and is thus preferably implemented by a semiconductor memorywhich has a relatively (for example, compared to the external memory1312) small capacity but has a high response speed, such as a staticrandom access memory (SRAM).

The codec engine 1516 performs a process regarding coding or decoding ofimage data. A coding or decoding method supported by the codec engine1516 is arbitrary, and the number thereof may be one, and may be plural.For example, the codec engine 1516 may have codec functions of aplurality of coding/decoding methods, and may perform coding of imagedata or decoding of coded data in a method selected from among themethods.

In the example illustrated in FIG. 93, the codec engine 1516 includes,for example, MPEG-2 Video 1541, AVC/H.264 1542, HEVC/H.265 1543,HEVC/H.265 (Scalable) 1544, HEVC/H.265 (Multi-view) 1545, and MPEG-DASH1551, as functional blocks of processes regarding codec.

The MPEG-2 Video 1541 is a functional block which codes or decodes imagedata in the MPEG-2 method. The AVC/H.264 1542 is a functional blockwhich codes or decodes image data in the AVC method. The HEVC/H.265 1543is a functional block which codes or decodes image data in the HEVCmethod. The HEVC/H.265 (Scalable) 1544 is a functional, block whichscalably codes or decodes image data in the HEVC method. HEVC/H.265(Multi-view) 1545 is a functional block which multi-view-codes ormulti-view-decodes image data in the HEVC method.

The MPEG-DASH 1551 is a functional block which transmits and receivesimage data in the MPEG-Dynamic Adaptive Streaming over HTTP (MPEG-DASH)method. The MPEG-DASH is a technique of performing streaming of a videoby using Hyper Text Transfer Protocol (HTTP), has one of features inwhich appropriate data is selected in the segment unit from among aplurality of coded data items which are prepared in advance and haveresolutions or the like different from each other, and is transmitted.The MPEG-DASH 1551 performs generation of a stream conforming to astandard, transmission control of the stream, or the like, and uses theabove-described MPEG-2 Video 1541, or HEVC/H.265 (Multi-view) 1545 forcoding/decoding of image data.

The memory interface 1517 is an interface for use in the external memory1312. Data supplied from the image processing engine 1514 or the codecengine 1516 is supplied to the external memory 1312 via the memoryinterface 1517. In addition, data read from the external memory 1312 issupplied to the video processor 1332 (the image processing engine 1514or the codec engine 1516) via the memory interface 1517.

The multiplexer/demultiplexer (MUX DEMUX) 1518 multiplexes ordemultiplexes various data items regarding an image, such as a bitstream of coded data, image data, and a video signal. A method ofmultiplexing and demultiplexing is arbitrary. For example, duringmultiplexing, the multiplexer/demultiplexer (MUX DEMUX) 1518 may notonly collect a plurality of data items into a single data item, but mayalso add predetermined header information or the like to the data. Inaddition, during demultiplexing, the multiplexer/demultiplexer (MUXDEMUX) 1518 may not only divide a single data item into a plurality of aplurality of data items, but may also add predetermined headerinformation or the like to each divided data item. In other words, themultiplexer/demultiplexer (MUX DEMUX) 1518 can convert a format of datathrough the multiplexing and demultiplexing. For example, themultiplexer/demultipiexer (MUX DEMUX) 1518 multiplexes a bit string soas to perform conversion into a transport stream which is a bit stringwith a transmission format or data (file data) with a recording fileformat. Of course, inverse conversion thereof can be performed throughdemultiplexing.

The network interface 1519 is an interface dedicated to, for example,the broadband modem 1333 or the connectivity 1321 (FIG. 91). The videointerface 1520 is an interface dedicated to, for example, theconnectivity 1321 or the camera 1322 (FIG. 91).

Next, an example of an operation of the video processor 1332 will bedescribed. For example, when a transport stream is received from anexternal network via, for example, the connectivity 1321 or thebroadband modem 1333 (FIG. 91), the transport stream is supplied to themultipiexer/demultiplexer (MUX DEMUX) 1518 via the network interface1519 so as to be demultiplexed, and is then decoded by the codec engine1516. Image data which is obtained through the coding in the codecengine 1516 undergoes a predetermined image process by, for example, theimage processing engine 1514 so as to undergo predetermined conversion,and is then is supplied to, for example, the connectivity 1321 (FIG. 91)via the display interface 1512, and an image thereof is displayed on amonitor. In addition, for example, the image data obtained through thecoding in the codec engine 1516 is decoded again by the codec engine1516 so as to be multiplexed by the multiplexer/demultiplexer (MUXDEMUX) 1518 and to be converted into file data, and is then output to,for example, the connectivity 1321 (FIG. 91) via the video interface1520 so as to be recorded on various recording media.

Further, for example, file data of coded data which is coded image dataand is read from a recording medium (not illustrated) by theconnectivity 1321 (FIG. 91) is supplied to the multiplexer/demultiplexer(MUX DEMUX) 1518 via the video interface 1520, and is then decoded bythe codec engine 1516. The image data obtained through the decoding inthe codec engine 1516 undergoes a predetermined image process by theimage processing engine 1514 so as to undergo predetermined conversionby the display engine 1513, and is then supplied to, for example, theconnectivity 1321 (FIG. 91) via the display interface 1512, and an imagethereof is displayed on the monitor. In addition, for example, the imagedata obtained through the decoding in the codec engine 1516 is codedagain by the codec engine 1516 so as to be multiplexed by themultiplexer/demultiplexer (MUX DEMUX) 1518 and to be converted into atransport stream, and is then output to, for example, the connectivity1321 or the broadband modem 1333 (FIG. 91) via the network interface1519 so as to be supplied to other apparatuses (not illustrated).

Further, transmission and reception of image data or other data betweenthe respective processing portions of the video processor 1332 areperformed by using, for example, the internal memory 151.5 or theexternal memory 1312. In addition, the power management module 1313controls the supply of power to, for example, the control portion 1511.

If the present technology is applied to the video processor 1332 havingthe configuration, the present technology related to each embodimentdescribed above may be applied to the codec engine 1516. In other words,for example, the codec engine 1516 may include a functional block forrealizing the coding device or the decoding device related to the firstembodiment. In addition, for example, if the codec engine 1516 includesthe above-described functional block, the video processor 1332 canachieve the same effects as the effects described with reference toFIGS. 6 to 13.

In addition, in the codec engine 1516, the present technology (that is,the function of the image coding device or the image decoding devicerelated to each embodiment described above) may be realized by hardwaresuch as a logic circuit, may be realized by software such as an embeddedprogram, and may be realized by both thereof.

As mentioned above, the two exemplary configurations of the videoprocessor 1332 have been described, but the video processor 1332 mayhave any configuration, and may have configurations other than the twoexemplary configurations. In addition, the video processor 1332 may beconfigured by a single semiconductor chip, and may be configured by aplurality of semiconductor chips. For example, a three-dimensionalstacked LSI in which a plurality of semiconductors are stacked may beused. Further, the video processor 1332 may be implemented by aplurality of LSIs.

(Application Examples to Apparatus)

The video set 1300 may be incorporated into various apparatuses whichprocess image data. For example, the video set 1300 may be incorporatedinto the television apparatus 900 (FIG. 84), the mobile phone 920 (FIG.85), the recording/reproducing apparatus 940 (FIG. 86), the imagingapparatus 960 (FIG. 87), and the like. The video set 1300 isincorporated into the apparatus, and thus the apparatus can achieve thesame effects as the effects described with reference to FIGS. 6 to 13.

In addition, the video set 1300 may be incorporated into, for example,the terminal apparatuses such as the personal computer 1004, the AVapparatus 1005, the tablet device 1006, and the mobile phone 1007 of thedata transmission system 1000 of FIG. 88, the broadcasting station 1101and the terminal apparatus 1102 of the data transmission system 1100 ofFIG. 89, the imaging apparatus 1201 and the scalable coded data storagedevice 1202 of the imaging system 1200 of FIG. 90, and the like. Thevideo set 1300 is incorporated into the apparatus, and thus theapparatus can achieve the same effects as the effects described withreference to FIGS. 6 to 13.

In addition, even if only some of the above-described configurations ofthe video set 1300 include the video processor 1332, the configurationscan be implemented as configurations to which the present technology isapplied. For example, only the video processor 1332 may be implementedas a video processor to which the present technology is applied. Inaddition, for example, as described above, the processor, the videomodule 1311, or the like indicated by the dotted line 1341 may beimplemented as a processor, a module, or the like to which the presenttechnology is applied. Further, a combination of the video module 1311,the external memory 1312, the power management module 1313, and thefront end module 1314 may be implemented as the video unit 1361 to whichthe present technology is applied. Any configuration can achieve thesame effects as the effects described with reference to FIGS. 6 to 13.

In other words, any configuration including the video processor 1332 canbe incorporated into various apparatuses which process image data in thesame manner as in the video set 1300. For example, the video processor1332, the processor indicated by the dotted line 1341, the video module1311, or the video unit 1361 can be incorporated into the televisionapparatus 900 (FIG. 84), the mobile phone 920 (FIG. 85), therecording/reproducing apparatus 940 (FIG. 86), the imaging apparatus 960(FIG. 87), the terminal apparatuses such as the personal computer 1004,the AV apparatus 1005, the tablet device 1006, and the mobile phone 1007of the data transmission system 1000 of FIG. 88, the broadcastingstation 1101 and the terminal apparatus 1102 of the data transmissionsystem 1100 of FIG. 89, the imaging apparatus 1201 and the scalablecoded data storage device 1202 of the imaging system 1200 of FIG. 90,and the like. Any one of configurations to which the present technologyis applied is incorporated into the apparatus, and thus the apparatuscan achieve the same effects as the effects described with reference toFIGS. 6 to 13 in the same manner as in the video set 1300.

In addition, in the present specification, description has been made ofan example in which various information pieces such as conversioninformation, DR conversion information, and an approximate knee pointindex are multiplexed into coded data, and are transmitted from a codingside to a decoding side. However, a method of transmitting theinformation is not limited to this example. For example, the informationmay be transmitted or recorded as separate data associated with codeddata without being multiplexed into the coded data. Here, the term“associated” indicates that an image (which may be a part of the image,such as a slice or a block) included in a bit stream is made to belinked to information corresponding to the image during decoding. Inother words, the information may be transmitted on a transmission pathdifferent from that of the coded data. In addition, the information maybe recorded on a recording medium (or a different recording area of thesame recording medium) different from that of the coded data. Further,the information and the coded data may be associated with each other inany unit such as a plurality of frames, a single frame, or a part of aframe.

In addition, in the present specification, the system indicates a set ofa plurality of constituent elements (devices, modules (components), orthe like), and it does not matter whether or not all constituentelements are located in the same casing. Therefore, a plurality ofdevices which are stored in separate casings and are connected to eachother via a network, a single device in which a plurality of modules arestored in a single casing, are all a system.

The effects disclosed in the present specification are only an exampleand are not limited, and there may be other effects.

In addition, embodiments of the present disclosure are not limited tothe above-described embodiments, and may have various modificationswithin the scope without departing from the spirit of the presentdisclosure.

For example, the present disclosure may have a cloud computingconfiguration in which a single function is distributed to a pluralityof devices via a network and is processed in cooperation with eachother.

Further, each step described in the above flowchart may be performed asingle device, and may also be performed by a plurality of devices in adistribution manner.

Furthermore, in a case where a plurality of processes are included in asingle step, the plurality of processes included in the single step maybe performed by a single device, and may also be performed by aplurality of devices in a distribution manner.

The present disclosure may have the following configurations.

(1) A decoding device including: circuitry configured to receive codeddata and conversion information, the coded data pertaining to an imagehaving luminance in a first dynamic range and the conversion informationpertaining to a conversion of dynamic range of the luminance of theimage from the first dynamic range into a second dynamic range; anddecode the received coded data so as to generate the image, wherein theconversion uses a knee function.

(2) The decoding device according to the above (1), wherein theconversion uses a knee point.

(3) The decoding device according to the above (1) or (2), wherein theconversion uses the knee function to map the dynamic range of theluminance of the image from the first dynamic range into the seconddynamic range, and the knee function is defined by the knee point.

(4) The decoding device according to any of the above (1) to (3),wherein the conversion information includes pre-conversion informationindicating a range of luminance which is a knee function target in thefirst dynamic range and post-conversion information indicating a rangeof luminance in the second dynamic range that corresponds to the rangeof luminance which is the knee function target in the first dynamicrange.

(5) The decoding device according to any of the above (1) to (4),wherein the pre-conversion information indicates the range of luminancewhich is converted by knee function at a same conversion ratio as aconversion range of the first dynamic range.

(6) The decoding device according to any of the above (1) to 5), whereinthe conversion uses the knee function which is defined by a plurality ofknee points.

(7) The decoding device according to any of the above (1) to (6),wherein the conversion information includes a plurality of pairs of thepre-conversion information and the post-conversion information.

(8) The decoding device according to any of the above (1) to (7),wherein the conversion uses the knee function by mapping the dynamicrange of the luminance of the image from the first dynamic range intothe second dynamic range, and a plurality of adjacent segments of thefirst dynamic range of the luminance are mapped to a correspondingplurality of adjacent segments of the second dynamic range of theluminance based on boundaries between adjacent segments defined by aplurality of knee points.

(9) The decoding device according to any of the above (1) to (8),wherein the conversion uses the knee function by mapping the dynamicrange of the luminance of the image from the first dynamic range intothe second dynamic range at a first conversion ratio to a point definedby the knee point and at a second conversion ratio from the pointdefined by the knee point.

(10) The decoding device according to any of the above (1) to (9),wherein the knee function is specified by an SEI message.

(11) The decoding device according to any of the above (1) to (10),wherein the SEI message includes a setting of a knee_function_id.

(12) A decoding method of causing a decoding device to perform:receiving coded data and conversion information, the coded datapertaining to an image having luminance in a first dynamic range and theconversion information pertaining to a conversion of dynamic range ofthe luminance of the image from the first dynamic range into a seconddynamic range; and decoding the received coded data so as to generatethe image, wherein the conversion uses a knee function.

(13) The decoding method according to the above (12), wherein theconversion information includes pre-conversion information indicating arange of luminance which is a knee function target in the first dynamicrange and post-conversion information indicating a range of luminance inthe second dynamic range that corresponds to the range of luminancewhich is the knee function target in the first dynamic range.

(14) The decoding method according to the above (12) or (13), whereinthe pre-conversion information indicates the range of luminance which isconverted by knee function at a same conversion ratio as a conversionrange of the first dynamic range.

(15) The decoding method according to any of the above (12) to (14),wherein the conversion information includes a plurality of pairs of thepre-conversion information and the post-conversion information.

(16) The decoding method according to any of the above (12) to (15),wherein the conversion uses the knee function by mapping the dynamicrange of the luminance of the image from the first dynamic range intothe second dynamic range at a first conversion ratio to a point definedby the knee point and at a second conversion ratio from the pointdefined by the knee point.

(17) A coding device including: circuitry configured to set conversioninformation pertaining to a conversion of dynamic range of a luminanceof an image from a first dynamic range into a second dynamic range; andcode the image having luminance in the first dynamic range so as togenerate coded data, wherein the conversion uses a knee function.

(18) The coding device according to the above (17), wherein theconversion information includes pre-conversion information indicating arange of luminance which is a knee function target in the first dynamicrange and post-conversion information indicating a range of luminance inthe second dynamic range that corresponds to the range of luminancewhich is the knee function target in the first dynamic range.

(19) The coding device according to the above (17) or (18), wherein thepre-conversion information indicates the range of luminance which isconverted by knee function at a same conversion ratio as a conversionrange of the first dynamic range.

(20) The coding device according to any of the above (17) to (19),wherein the conversion information includes a plurality of pairs of thepre-conversion information and the post-conversion information.

(21) The coding device according to any of the above (17) to (20),wherein the conversion uses the knee function by mapping the dynamicrange of the luminance of the image from the first dynamic range intothe second dynamic range at a first conversion ratio to a point definedby the knee point and at a second conversion ratio from the pointdefined by the knee point.

(22) A non-transitory computer-readable medium having stored thereoncoded data and conversion information, the coded data pertaining to animage having luminance in a first dynamic range and the conversioninformation pertaining to a conversion of dynamic range of the luminanceof the image from the first dynamic range into a second dynamic range,wherein a decoding device decodes coded data, generates the image basedon the decoded data, and converts the dynamic range based on theconversion information including a knee point.

(23) The non-transitory computer-readable medium according to the above(22), wherein the conversion information includes pre-conversioninformation indicating a range of luminance which is a knee functiontarget in the first dynamic range and post-conversion informationindicating a range of luminance in the second dynamic range thatcorresponds to the range of luminance which is the knee function targetin the first dynamic range.

(24) The non-transitory computer-readable medium according to the above(22) or (23), wherein the pre-conversion information indicates the rangeof luminance which is converted by knee function at a same conversionratio as a conversion range of the first dynamic range.

(25) The non-transitory computer-readable medium according to any of theabove (22) to (24), wherein the conversion information includes aplurality of pairs of the pre-conversion information and thepost-conversion information.

(26) The non-transitory computer-readable medium according to any of theabove (22) to (25), wherein the conversion uses the knee function bymapping the dynamic range of the luminance of the image from the firstdynamic range into the second dynamic range at a first conversion ratioto a point defined by the knee point and at a second conversion ratiofrom the point defined by the knee point.

(27) A decoding device including an extraction unit that extracts codeddata and conversion information from a coded stream including the codeddata of a first image which is an image having luminance in a firstdynamic range and the conversion information regarding conversion of adynamic range of the luminance of the image from the first dynamic rangeinto a second dynamic range; and a decoding unit that decodes the codeddata extracted by the extraction unit so as to generate the first image.

(28) The decoding device according to the above (27), further includinga conversion unit that converts the first image which is generated bythe decoding unit into a second image which is the image havingluminance in the second dynamic range on the basis of the conversioninformation extracted by the extraction unit.

(29) The decoding device according to the above (27) or (28), in whichthe conversion is performed by knee-converting the luminance of thefirst image.

(30) The decoding device according to the any one of the above (27) to(29), in which the conversion information includes pre-conversioninformation indicating a range of luminance which is a knee conversiontarget in the first dynamic range and post-conversion informationindicating a range of luminance in the second dynamic range,corresponding to the range.

(31) The decoding device according to the any one of the above (27) to(30), in which the pre-conversion information indicates a range ofluminance which is knee-converted at the same conversion ratio as aconversion range of the first dynamic range, and in which the conversioninformation includes a plurality of pairs of the pre-conversioninformation and the post-conversion information.

(32) The decoding device according to the any one of the above (27) to(31), further including a selection unit that selects a predeterminednumber pairs from among the plurality of pairs included in theconversion information which is extracted by the extraction unit, in anorder in which the pairs are included in the conversion information.

(33) The decoding device according to the any one of the above (27) to(31), further including a selection unit that selects a predeterminednumber pairs from among the plurality of pairs included in theconversion information on the basis of priority information indicatingan order in which a priority of the pair is higher, in which theextraction unit extracts the priority information included in the codedstream.

(34) The decoding device according to the any one of the above (27) to(33), further including a transmission unit that transmits thepredetermined number of pairs selected by the selection unit.

(35) The decoding device according to any one of the above (27) to (34),in which the conversion information includes at least one of a maximumvalue of the luminance of the first image and a maximum value of theluminance of the second image.

(36) The decoding device according to any one of the above (27) to (35),in which the conversion information includes at least one of an expectedvalue of brightness of a display unit which displays the first image andan expected value of brightness of a display unit which displays thesecond image.

(37) A decoding method of causing a decoding device to performextracting coded data and conversion information from a coded streamincluding the coded data of a first image which is an image havingluminance in a first dynamic range and the conversion information whichis information regarding conversion of a dynamic range of the luminanceof the image from the first dynamic range into a second dynamic range;and decoding the extracted coded data so as to generate the first image.

(38) A coding device including a setting unit that sets conversioninformation which is information regarding conversion of a dynamic rangeof luminance of an image from a first dynamic range into a seconddynamic range; a coding unit that codes a first image which is the imagehaving luminance in the first dynamic range so as to generate codeddata; and a transmission unit that transmits a coded stream includingthe conversion information set by the setting unit and the coded data ofthe first image generated by the coding unit.

(39) The coding device according to the above (38), in which theconversion is performed by knee-converting the luminance of the firstimage.

(40) The coding device according to the above (38) or (39), in which theconversion information includes pre-conversion information indicating arange of luminance which is a knee conversion target in the firstdynamic range and post-conversion information indicating a range ofluminance in the second dynamic range, corresponding to the range.

(41) The coding device according to any one of the above (38) to (40),in which the pre-conversion information indicates a range of luminancewhich is knee-converted at the same conversion ratio as a conversionrange of the first dynamic range, and in which the conversioninformation includes a plurality of pairs of the pre-conversioninformation and the post-conversion information.

(42) The coding device according to any one of the above (38) to (41),in which the conversion information includes the plurality of pairs ofpre-conversion information and post-conversion information in an orderin which a priority is higher.

(43) The coding device according to any one of the above (38) to (42),in which the transmission unit transmits priority information indicatingan order in which a priority of the pair is higher.

(44) The coding device according to any one of the above (38) to (43),in which the conversion information includes at least one of a maximumvalue of the luminance of the first image and a maximum value of theluminance of the second image.

(45) The coding device according to any one of the above (38) to (44),in which the conversion information includes at least one of an expectedvalue of brightness of a display unit which displays the first image andan expected value of brightness of a display unit which displays thesecond image.

(46) A coding method of causing a coding device to perform settingconversion information which is information regarding conversion of adynamic range of luminance of an image from a first dynamic range into asecond dynamic range; coding a first image which is the image havingluminance in the first dynamic range so as to generate coded data; andtransmitting a coded stream including the set conversion information andthe generated coded data of the first image.

It should be understood by those skilled in the art that variousmodifications, combinations, sub-combinations and alterations may occurdepending on design requirements and other factors insofar as they arewithin the scope of the appended claims or the equivalents thereof.

REFERENCE SIGNS LIST

-   10 Coding device-   11 Setting unit-   12 Coding unit-   13 Transmission unit-   14 Conversion unit-   50 Decoding device-   52 Extraction unit-   53 Decoding unit-   54 Conversion unit-   70 Coding device-   71 Setting unit-   72 Coding unit-   90 Decoding device-   91 Extraction unit-   92 Decoding unit-   93 Conversion unit-   110 Decoding system-   111 Decoding device-   112 Display device-   121 Selection unit-   122 Transmission unit

The invention claimed is:
 1. A decoding device comprising: circuitryconfigured to receive coded data and conversion information, the codeddata pertaining to an image having luminance in a first dynamic rangeand the conversion information pertaining to a conversion of dynamicrange of the luminance of the image from the first dynamic range into asecond dynamic range; and decode the received coded data so as togenerate the image, wherein the conversion uses a knee function that isdefined by the connecting of a plurality of knee points in a set order,each knee point of the knee function being located at a respectivecoordinate that is defined by a respective input knee point and arespective output knee point, the respective input knee point specifyinga luminance level of the knee point at the respective coordinate for theimage of the first dynamic range, and the respective output knee pointspecifying a luminance level of the knee point at the respectivecoordinate for the image of the second dynamic range after theconversion.
 2. The decoding device according to claim 1, wherein theconversion uses the knee function to map the dynamic range of theluminance of the image from the first dynamic range into the seconddynamic range.
 3. The decoding device according to claim 2, wherein theconversion information includes pre-conversion information andpost-conversion information, wherein the pre-conversion informationindicates a range of luminance in the first dynamic range, and whereinthe post-conversion information indicates a range of luminance in thesecond dynamic range that corresponds to the range of luminanceindicated by the pre-conversion information.
 4. The decoding deviceaccording to claim 3, wherein the pre-conversion information indicatesthe range of luminance which is converted by knee function at a sameconversion ratio as a conversion range of the first dynamic range. 5.The decoding device according to claim 3, wherein the conversioninformation includes a plurality of pairs of the pre-conversioninformation and the post-conversion information.
 6. The decoding deviceaccording to claim 1, wherein the conversion uses the knee function bymapping the dynamic range of the luminance of the image from the firstdynamic range into the second dynamic range, and a plurality of adjacentsegments of the first dynamic range of the luminance are mapped to acorresponding plurality of adjacent segments of the second dynamic rangeof the luminance based on boundaries between adjacent segments definedby the plurality of knee points.
 7. The decoding device according toclaim 2, wherein the conversion uses the knee function by mapping thedynamic range of the luminance of the image from the first dynamic rangeinto the second dynamic range at a first conversion ratio to a pointdefined by the at least one knee point and at a second conversion ratiofrom the point defined by the at least one knee point.
 8. The decodingdevice according to claim 1, wherein the knee function is specified by aSupplemental Enhancement Information (SEI) message.
 9. The decodingdevice according to claim 8, wherein the SEI message includes a settingof a knee_function_id.
 10. A decoding method of causing a decodingdevice to perform: receiving coded data and conversion information, thecoded data pertaining to an image having luminance in a first dynamicrange and the conversion information pertaining to a conversion ofdynamic range of the luminance of the image from the first dynamic rangeinto a second dynamic range; and decoding the received coded data so asto generate the image, wherein the conversion uses a knee function thatis defined by the connecting of a plurality of knee points in a setorder, each knee point of the knee function being located at arespective coordinate that is defined by a respective input knee pointand a respective output knee point, the respective input knee pointspecifying a luminance level of the knee point at the respectivecoordinate for the image of the first dynamic range, and the respectiveoutput knee point specifying a luminance level of the knee point at therespective coordinate for the image of the second dynamic range afterthe conversion.
 11. The decoding method according to claim 10, whereinthe conversion information includes pre-conversion informationindicating a range of luminance in the first dynamic range andpost-conversion information indicating a range of luminance in thesecond dynamic range that corresponds to the range of luminanceindicated by the pre-conversion information.
 12. The decoding methodaccording to claim 11, wherein the pre-conversion information indicatesthe range of luminance which is converted by knee function at a sameconversion ratio as a conversion range of the first dynamic range. 13.The decoding method according to claim 11, wherein the conversioninformation includes a plurality of pairs of the pre-conversioninformation and the post-conversion information.
 14. The decoding methodaccording to claim 10, wherein the conversion uses the knee function bymapping the dynamic range of the luminance of the image from the firstdynamic range into the second dynamic range at a first conversion ratioto a point defined by at least one knee point of the plurality of kneepoints and at a second conversion ratio from the point defined by the atleast one knee point.
 15. A coding device comprising: circuitryconfigured to set conversion information pertaining to a conversion ofdynamic range of a luminance of an image from a first dynamic range intoa second dynamic range; and code the image having luminance in the firstdynamic range so as to generate coded data, wherein the conversion usesa knee function that is defined by the connecting of a plurality of kneepoints in a set order, each knee point of the knee function beinglocated at a respective coordinate that is defined by a respective inputknee point and a respective output knee point, the respective input kneepoint specifying a luminance level of the knee point at the respectivecoordinate for the image of the first dynamic range, and the respectiveoutput knee point specifying a luminance level of the knee point at therespective coordinate for the image of the second dynamic range afterthe conversion.
 16. The coding device according to claim 15, wherein theconversion information includes pre-conversion information indicating arange of luminance in the first dynamic range and post-conversioninformation indicating a range of luminance in the second dynamic rangethat corresponds to the range of luminance indicated by thepre-conversion information.
 17. The coding device according to claim 16,wherein the pre-conversion information indicates the range of luminancewhich is converted by knee function at a same conversion ratio as aconversion range of the first dynamic range.
 18. The coding deviceaccording to claim 16, wherein the conversion information includes aplurality of pairs of the pre-conversion information and thepost-conversion information.
 19. The decoding device according to claim1, wherein the conversion information includes conversion persistenceinformation which is information indicating whether or not theconversion information is applied to a plurality of continuous pictures.20. The decoding device according to claim 1, wherein the plurality ofknee points specify conversion ratios at respective points of the kneefunction used for the conversion of the dynamic range.
 21. The decodingdevice according to claim 1, wherein each knee point of the plurality ofknee points specifies a coordinate position that associates a respectivevalue of luminance of the image before conversion to a respective valueof luminance of the image after conversion.
 22. A non-transitorycomputer-readable medium having stored thereon coded data and conversioninformation, the coded data pertaining to an image having luminance in afirst dynamic range and the conversion information pertaining to aconversion of dynamic range of the luminance of the image from the firstdynamic range into a second dynamic range, wherein when a decodingdevice decodes the coded data, the decoding device is caused to generatethe image based on the decoded data, and also caused to convert, using aknee function that is defined by the connecting of a plurality of kneepoints in a set order, the dynamic range of the luminance of the imagefrom the first dynamic range into a second dynamic range based on theconversion information, each knee point of the knee function beinglocated at a respective coordinate that is defined by a respective inputknee point and a respective output knee point, the respective input kneepoint specifying a luminance level of the knee point at the respectivecoordinate for the image of the first dynamic range, and the respectiveoutput knee point specifying a luminance level of the knee point at therespective coordinate for the image of the second dynamic range afterthe conversion.
 23. The decoding device according to claim 1, whereinthe knee function is defined by connecting the knee points to each otherin an ascending order.
 24. The decoding device according to claim 1,wherein data processing requirement is minimized by defining the kneefunction utilizing relationship between adjacent knee points.